Search

We study the optimal consumption, investment and lifeinsurance purchase and selection strategies for a wage-earner with an uncertain lifetime with access to a financial market comprised of one risk-free security and one risky-asset whose prices evolve according to linear diffusions modulated by a continuous-time stochastic process determined by an additional diffusive nonlinear stochastic differential equation. The process modulating the linear diffusions may be regarded as an indicator describing the state of the economy in a given instant of time. Additionally, we allow the Brownian motions driving each of these equations to be correlated. The lifeinsurance market under consideration herein consists of a fixed number of providers offering pairwise distinct contracts. We use dynamic programming techniques to characterize the solutions to the problem described above for a general family of utility functions, studying the case of discounted constant relative risk aversion utilities with more detail. [ABSTRACT FROM AUTHOR]

We study an exit contract design problem, where one provides a universal exit contract to multiple heterogeneous agents, with which each agent chooses an optimal (exit) stopping time. The problem consists in optimizing the universal exit contract w.r.t. some criterion depending on the contract as well as the agents' exit times. Under a technical monotonicity condition, and by using Bank-El Karoui's representation of stochastic processes, we are able to transform the initial contract optimization problem into an optimal control problem. The latter is also equivalent to an optimal multiple stopping problem and the existence of the optimal contract is proved. We next show that the problem in the continuous-time setting can be approximated by a sequence of discrete-time ones, which would induce a natural numerical approximation method. We finally discuss the optimization problem over the class of all Markovian and/or continuous exit contracts. [ABSTRACT FROM AUTHOR]

We present a pair of adjoint optimal control problems characterizing a class of time-symmetric stochastic processes defined on random time intervals. The associated PDEs are of free-boundary type. The particularity of our approach is that it involves two adjoint optimal stopping times adapted to a pair of filtrations, the traditional increasing one and another, decreasing. They are the keys of the time symmetry of the construction, which can be regarded as a generalization of 'Schrödinger's problem' (1931–1932) to space-time domains. The relation with the notion of 'Hidden diffusions' is also described. [ABSTRACT FROM AUTHOR]

A controlled discrete-time stochastic process is proposed as a model for the state of a machine. In the proposed model, normal and random wear of the machine are considered. The random wear of the machine during a time unit can be either a discrete or a continuous random variable. The objective is to find a control policy that maximises the profits generated by the machine over its useful lifetime. In this problem, the optimiser must decide whether to do or not to do the maintenance work on the machine at each time unit. The significance of the paper is that the final time in the optimal control problem is a random variable denoting the first time the machine must be replaced. To obtain the optimal control, one can try to solve the dynamic programming equation, which reduces to a difference or an integral equation, satisfied by the value function. Finally, particular cases are solved exactly, or approximately, and explicitly to demonstrate and validate the proposed model. [ABSTRACT FROM AUTHOR]

This article studies the formation mission design problem for commercial aircraft in the presence of uncertainties. Specifically, it considers uncertainties in the departure times of the aircraft and in the fuel burn savings for the trailing aircraft. Given several commercial flights, the problem consists in arranging them in formation or solo flights and finding the trajectories that minimize the expected value of the DOC of the flights. The formation mission design problem is formulated as an optimal control problem of a stochastic switched dynamical system and solved using nonintrusive gPC-based stochastic collocation. The stochastic collocation method converts the SSOCP into an augmented deterministic switched optimal control problem. With this approach, a small number of sample points of the random parameters are used to jointly solve particular instances of the switched optimal control problem. The obtained solutions are then expressed as orthogonal polynomial expansions in terms of the random parameters using these sample points. This technique allows statistical and global sensitivity analysis of the stochastic solutions to be conducted at a low computational cost. The aim of this article is to establish if, in the presence of uncertainties, a formation mission is beneficial with respect to solo flight in terms of the expected value of the direct operating costs. Several numerical experiments have been conducted in which uncertainties on the departure times and on the fuel saving during formation flight have been considered. The obtained results demonstrate that benefits can be achieved even in the presence of these uncertainties. [ABSTRACT FROM AUTHOR]

Stochastic flows of an advective-diffusive nature are ubiquitous in biology and the physical sciences. Of particular interest is the problem of reconciling observed marginal distributions with a given prior posed by Schrödinger in 1932 and known as the Schrödinger Bridge Problem (SBP). It turns out that Schrödinger's problem can be viewed as both a modeling problem and a control problem. Due to their fundamental significance, the SBP and its deterministic (zero-noise limit) counterpart of Optimal Mass Transport (OMT) have recently received interest within a broad spectrum of disciplines, including physics, stochastic control, computer science, probability theory, and geometry. Yet, while the mathematics and applications of the SBP/OMT have been developing at a considerable pace, accounting for marginals of unequal mass has received scant attention; the problem of interpolating between "unbalanced" marginals has been approached by introducing source/sink terms into the transport equations, in an ad hoc manner, chiefly driven by applications in image registration. Nevertheless, losses are inherent in many physical processes, and thereby models that account for lossy transport may also need to be reconciled with observed marginals following Schrödinger's dictum; that is, it is necessary to adjust the probability of trajectories of particles, including those that do not make it to the terminal observation point, so that the updated law represents the most likely way that particles may have been transported, or have vanished, at some intermediate point. Thus, the purpose of this work is to develop such a natural generalization of the SBP for stochastic evolution with losses, whereupon particles are "killed" (jump into a coffin/extinction state) according to a probabilistic law, and thereby mass is gradually lost along their stochastically driven flow. Through a suitable embedding we turn the problem into an SBP for stochastic processes that combine diffusive and jump characteristics. Then, following a large-deviations formalism in the style of Schrödinger, given a prior law that allows for losses, we ask for the most probable evolution of particles along with the most likely killing rate as the particles transition between the specified marginals. Our approach differs sharply from previous work involving a Feynman-Kac multiplicative reweighing of the reference measure: The latter, as we argue, is far from Schrödinger's quest. An iterative scheme, generalizing the celebrated Fortet-IPF-Sinkhorn algorithm, permits us to compute the new drift and the new killing rate of the path-space solution measure. We finally formulate and solve a related fluid-dynamic control problem for the flow of one-time marginals where both the drift and the new killing rate play the role of control variables. [ABSTRACT FROM AUTHOR]

In this paper, inspired by various stochastic linear-quadratic (LQ, for short) problems, we develop the method of continuation to study nonlinear forward–backward stochastic differential equations (FBSDEs, for short) in a kind of domination-monotonicity frameworks. The coupling of FBSDEs is in a general form, i.e., it not only appears in integral terms and terminal terms, but also in initial terms. By virtue of introducing various matrices, matrix-valued random variables and matrix-valued stochastic processes, we present the domination-monotonicity framework carefully and rigorously. A unique solvability result and a pair of estimates for coupled FBSDEs are obtained (see Theorem 3.5 in the case of simple domination-monotonicity conditions and Theorem 5.2 in the case of multi-level self-similar domination-monotonicity structures). As applications of theoretical results, the related stochastic Hamiltonian systems of several LQ problems are discussed. [ABSTRACT FROM AUTHOR]

Based on a generative model (GM) and beliefs over hidden states, the free energy principle (FEP) enables an agent to sense and act by minimizing a free energy bound on Bayesian surprise, i.e., the negative logarithm of the marginal likelihood. Inclusion of desired states in the form of prior beliefs in the GM leads to active inference (ActInf). In this work, we aim to reveal connections between ActInf and stochastic optimal control. We reveal that, in contrast to standard cost and constraint-based solutions, ActInf gives rise to a minimization problem that includes both an information-theoretic surprise term and a model-predictive control cost term. We further show under which conditions both methodologies yield the same solution for estimation and control. For a case with linear Gaussian dynamics and a quadratic cost, we illustrate the performance of ActInf under varying system parameters and compare to classical solutions for estimation and control. [ABSTRACT FROM AUTHOR]

This paper presents a stochastic framework for the optimization of microgrids that has the functionality of providing flexibility services to System Operators (SOs) considering uncertainties in the energy forecast. The methodology is developed with the aim of being applied to complex microgrids composed of different distributed energy resources and hybrid energy storage systems (ESS). The associated optimization problem is operated in two stages: the first one performs a stochastic optimization of the microgrid in order to reserve an up/down regulation capacity with which to deal with the energy forecast uncertainties of the microgrid. The different microgrid devices are optimized by considering their operational costs in order to achieve their optimal operation in the Day-Ahead Market (DM). The second stage is used to re-schedule the initial planning according to the signal request and an economic offer from the SO. The control problem is developed using Stochastic Model Predictive Control (SMPC) techniques and Mixed-Integer Quadratic Programming (MIQP), owing to the presence of logic, integer, mixed and probabilistic variables. The simulation results show that the proposed methodology reduces the risk of undergoing up/down-penalty deviations in the Regulation Service Market (RM), also being able to provide flexibility services to the SOs, despite being subject to uncertainties in the energy forecast carried out for the microgrid. [ABSTRACT FROM AUTHOR]

We study the optimal control problem for a descriptor system whose evolution is described by Ito's differential-algebraic equation. The quadratic cost functional is considered. The main constraint is that the characteristic matrix pencil corresponding to the equation is regular. We establish the conditions for the existence and uniqueness of the optimal control and the corresponding optimal state. The results are illustrated on an example of a descriptor system that describes transient states in a radio engineering filter with random perturbations in the form of white noise. [ABSTRACT FROM AUTHOR]

Control based on vision data is a growing field of research and it is widespread in industry. The amount of data in each image and the processing needed to obtain control-relevant information from this data lead to significant delays with a large variability in the control loops. This often causes performance deterioration since in many cases the delay variability is not explicitly addressed in the control design. In this paper, we approach this problem by applying the ideas of recently developed model-based control design methods, which are tailored to address stochastic delays directly, to the motion control of an omnidirectional robot with a vision-based self-localization algorithm. The completion time or delay of the Random Sample Consensus (RANSAC) based localization algorithm is identified as a stochastic random variable with significant variability, illustrating the practical difficulties with data processing. Our main aim is to show that the novel deadline-driven and event-driven control designs significantly outperform a traditional periodic control implementation for a stochastic optimal control performance index. [ABSTRACT FROM AUTHOR]

In the Paris subway system, stations represent about one-third of the overall energy consumption. Within stations, ventilation is among the top consuming devices; it is operated at maximum airflow all day long, for air quality reasons. In this paper, we present a concept of energy system that displays comparable air quality while consuming much less energy. The system comprises a battery that makes it possible to recover the trains braking energy, arriving under the form of erratic and strong peaks. We propose an energy management system that, at short time scale, controls energy flows and ventilation airflow. By using proper optimization algorithms, we manage to match supply with demand, while minimizing energy daily costs. For this purpose, we have designed algorithms that take into account the braking variability. They are based on the so-called stochastic dynamic programming (SDP) mathematical framework. We fairly compare SDP-based algorithms with the widespread model predictive control (MPC) ones. First, both SDP and MPC yield energy/money operating savings of the order of one-third, compared to the current management without battery. Second, depending on the specific design, we observe that SDP outperforms MPC by a few percent, with an easier online numerical implementation. [ABSTRACT FROM AUTHOR]

Abstract This paper addresses the problem of controlling a Markov chain so as to minimize the long-run expected average cost per unit time when the invariant distribution is unknown but we know it belongs to a given uncertain set. The mathematical model used to describe this set is the total variation distance uncertainty. We show that the equilibrium control policy, which yields higher probability to the states with low cost and lower probability to the states with the high cost, is an optimal control policy that minimizes the average cost. Recognition of such a policy may be of value in practical situations with constraints consistent to those studied here when the invariant distribution is uncertain and deriving online an optimal control policy is required. [ABSTRACT FROM AUTHOR]

Neural approximations of the optimal stationary closed-loop control strategies for discounted infinite-horizon stochastic optimal control problems are investigated. It is shown that for a family of such problems, the minimal number of network parameters needed to achieve a desired accuracy of the approximate solution does not grow exponentially with the number of state variables. In such a way, neural-network approximation mitigates the so-called “curse of dimensionality”. The obtained theoretical results point out the potentialities of neural-network based approximation in the framework of sequential decision problems with continuous state, control, and disturbance spaces. [ABSTRACT FROM AUTHOR]

As the application of probabilistic models in robotic applications increases, a systematic robot control approach considering the effects of uncertainty becomes indispensable. Inspired by human sensorimotor findings, in this paper, we study the stochastic optimal control problem with high-order cost statistics in order to synthesize uncertainty-dependent actions in robotic scenarios with multiple uncertainty sources. We present locally optimal risk-sensitive and cost-cumulant solutions for settings with nonlinear dynamics, multiple additive uncertainty sources, and nonquadratic costs. The influence of each uncertainty source on the cost can be individually parameterized offering additional flexibility in the control design. We further analyze the case in which the static uncertain parameters are involved. The simulations of several linear and nonlinear settings with nonquadratic costs and an experiment on a real robotic platform validate our approach and illustrate its peculiarities. [ABSTRACT FROM AUTHOR]

In this study, the problem of designing a stochastic optimal controller for sampled‐data systems whose sampling interval is subjected to a certain probability distribution is addressed. To design the controller, the Kronecker product operation and the Vandermonde matrix were introduced. A design method of the stochastic optimal controller is proposed. It is shown that the controller guarantee that the closed‐loop system has exponentially mean square stability. Finally, the simulation results illustrate the effectiveness and practicability of the proposed method. [ABSTRACT FROM AUTHOR]

This work deals with a finite-horizon covariance control problem for discrete-time, stochastic linear systems with complete state information subject to input constraints. First, we present the main steps for the transcription of the covariance control problem, which is originally formulated as a stochastic optimal control problem, into a deterministic nonlinear program (NLP) with a convex performance index and with both convex and non-convex constraints. In particular, the convex constraints in this nonlinear program are induced by the input constraints of the stochastic optimal control problem, whereas the non-convex constraints are induced by the requirement that the terminal state covariance be equal to a prescribed positive definite matrix. Subsequently, we associate this nonlinear program, via a simple convex relaxation technique, with a (convex) semi-definite program, which can be solved numerically by means of modern computational tools of convex optimization. Although, in general, the endpoints of a representative sample of closed-loop trajectories generated by the control policy that corresponds to the solution of the relaxed convex program are not expected to follow exactly the goal terminal Gaussian distribution, they are more likely to be concentrated near the mean of this distribution than if they were drawn from the latter, which is a desirable feature in practice. Numerical simulations that illustrate the key ideas of this work are also presented. [ABSTRACT FROM AUTHOR]

In this article, we study and compare two approaches to solving stochastic optimal control problems with an expectation constraint on the final state. The case of a probability constraint is included in this framework. The first approach is based on a dynamic programming principle and the second one uses Lagrange relaxation. In this article, we focus on discrete-time problems, but the two discussed approaches can be applied to discretized continuous-time problems. [ABSTRACT FROM AUTHOR]

We propose a computational scheme for the solution of the so-called minimum variance control problem for discrete-time stochastic linear systems subject to an explicit constraint on the 2-norm of the input (random) sequence. In our approach, we utilize a state space framework in which the minimum variance control problem is interpreted as a finite-horizon stochastic optimal control problem with incomplete state information. We show that if the set of admissible control policies for the stochastic optimal control problem consists exclusively of sequences of causal (non-anticipative) control laws that can be expressed as linear combinations of all the past and present outputs of the system together with its past inputs, then the stochastic optimal control problem can be reduced to a deterministic, finite-dimensional optimization problem. Subsequently, we show that the latter optimization problem can be associated with an equivalent convex program and in particular, a quadratically constrained quadratic program (QCQP), by means of a bilinear transformation. Finally, we present numerical simulations that illustrate the key ideas of this work. [ABSTRACT FROM AUTHOR]

In many practical applications, stability with respect to part of the system's states is often necessary with finite-time convergence to the equilibrium state of interest. Finite-time partial stability involves dynamical systems whose part of the trajectory converges to an equilibrium state in finite time. In this paper, we address finite-time partial stability in probability and uniform finite-time partial stability in probability for nonlinear stochastic dynamical systems. Specifically, we provide Lyapunov conditions involving a Lyapunov function that is positive definite and decrescent with respect to part of the system state and satisfies a differential inequality involving fractional powers for guaranteeing finite-time partial stability in probability. In addition, we show that finite-time partial stability in probability leads to uniqueness of solutions in forward time and we establish necessary and sufficient conditions for almost sure continuity of the settling-time operator of the nonlinear stochastic dynamical system. Finally, we develop a unified framework to address the problem of optimal nonlinear analysis and feedback control design for finite-time partial stochastic stability and finite-time, partial-state stochastic stabilization. Finite-time partial stability in probability of the closed-loop nonlinear system is guaranteed by means of a Lyapunov function that is positive definite and decrescent with respect to part of the system state and can clearly be seen to be the solution to the steady-state form of the stochastic Hamilton-Jacobi-Bellman equation guaranteeing both finite-time, partial-state stability and optimality. The overall framework provides the foundation for extending stochastic optimal linear-quadratic controller synthesis to nonlinear-nonquadratic optimal finite-time, partial-state stochastic stabilization. [ABSTRACT FROM AUTHOR]

This paper studies the first order backward stochastic partial differential equations suggested earlier for a case of multidimensional state domain with a boundary. These equations represent analogues of Hamilton--Jacobi--Bellman equations and allow one to construct the value function for stochastic optimal control problems with unspecified dynamics where the underlying processes do not necessarily satisfy stochastic differential equations of a known kind with a given structure. The problems considered arise in financial modeling. [ABSTRACT FROM AUTHOR]

This paper is concerned with the existence and uniqueness of solution for the optimal control problem governed by the stochastic FitzHugh–Nagumo equation driven by a Gaussian noise. First-order conditions of optimality are also obtained. [ABSTRACT FROM AUTHOR]

One of the measures for saving energy is the implementation of control strategies that reduce energy consumption during the machine idle periods. This paper proposes a framework that integrates different control policies for switching the machine off when the production is not critical, and on when the part flow has to be resumed. A general policy is formalized by modeling explicitly the power consumed in each system state. With this policy, the service can be resumed according to time threshold and when N parts have accumulated in the buffer. The behavior of the control strategy under different scenarios is numerically evaluated. Numerical results are based on data acquired with dedicated experimental measurements on a real machining center. A comparison with the common practice in manufacturing and with control policies that do not consider buffer information is also reported. [ABSTRACT FROM AUTHOR]

In this paper, we study a two-person zero-sum linear-quadratic stochastic differential game problem. From a new viewpoint, we construct an optimal feedback control-strategy pair for the game in a closed-loop form based on the solution of a Riccati equation. A key part of our analysis involves proving the global solvability of this Riccati equation, which is interesting in its own right. Moreover, we demonstrate an indefinite phenomenon arising from the linear-quadratic game. [ABSTRACT FROM AUTHOR]

The advent of networked control systems urges the digital control design to incorporate communication constraints efficiently. In order to accommodate this requirement, this article studies the joint design of controller and event-trigger for linear stochastic systems in the presence of a resource-limited communication channel which exhibits packet dropouts and time-delay. The event-trigger situated at the sensor decides at every sampling instance, whether to send information over the communication channel to the controller. The design approach is formulated as a stochastic average-cost optimization problem, where the communication constraints are reflected as an additional cost penalty of the average transmission rate. Different conditions on the communication model are given where the joint optimal design can be split into a separate control and event-trigger design. Based on these results, two suboptimal design approaches are developed. By using drift criteria, stability guarantees of the closed-loop system for both approaches are derived in terms of bounded moment stability. Numerical simulations illustrate the efficacy of the event-triggered approach compared with optimal time-triggered controllers. [ABSTRACT FROM AUTHOR]

A time-delayed stochastic optimal bounded control strategy for strongly non-linear systems under wide-band random excitations with actuator saturation is proposed based on the stochastic averaging method and the stochastic maximum principle. First, the partially averaged Itô equation for the system amplitude is derived by using the stochastic averaging method for strongly non-linear systems. The time-delayed feedback control force is approximated by a control force without time delay based on the periodically random behavior of the displacement and velocity of the system. The partially averaged Itô equation for the system energy is derived from that for the system amplitude by using Itô formula and the relation between system amplitude and system energy. Then, the adjoint equation and maximum condition of the partially averaged control problem are derived based on the stochastic maximum principle. The saturated optimal control force is determined from maximum condition and solving the forward–backward stochastic differential equations (FBSDEs). For infinite time-interval ergodic control, the adjoint variable is stationary process and the FBSDE is reduced to a ordinary differential equation. Finally, the stationary probability density of the Hamiltonian and other response statistics of optimally controlled system are obtained from solving the Fokker–Plank–Kolmogorov (FPK) equation associated with the fully averaged Itô equation of the controlled system. For comparison, the optimal control forces obtained from the time-delayed bang–bang control and the control without considering time delay are also presented. An example is worked out to illustrate the proposed procedure and its advantages. [ABSTRACT FROM AUTHOR]

A stochastic minimax optimal control strategy for uncertain quasi-Hamiltonian systems is proposed based on the stochastic averaging method, stochastic maximum principle and stochastic differential game theory. First, the partially completed averaged Itô stochastic differential equations are derived from a given system by using the stochastic averaging method for quasi-Hamiltonian systems with uncertain parameters. Then, the stochastic Hamiltonian system for minimax optimal control with a given performance index is established based on the stochastic maximum principle. The worst disturbances are determined by minimizing the Hamiltonian function, and the worst-case optimal controls are obtained by maximizing the minimal Hamiltonian function. The differential equation for adjoint process as a function of system energy is derived from the adjoint equation by using the Itô differential rule. Finally, two examples of controlled uncertain quasi-Hamiltonian systems are worked out to illustrate the application and effectiveness of the proposed control strategy. [ABSTRACT FROM AUTHOR]

Abstract: The paper considers a problem of stochastic control and dynamics of a single-degree-of-freedom system with piecewise linear stiffness subjected to combined periodic and white noise external excitations. To minimize the system response energy a bounded in magnitude control force is applied to the systems. The stochastic optimal control problem is handled through the dynamic programming approach. Based on the solution to the Hamilton–Jacobi–Bellman equation it is proposed to use the dry friction control law in the non-resonant case. In the resonant case the stochastic averaging procedure has been used to derive stochastic differential equations for system response amplitude and phase. The joint PDF of response amplitude and phase is derived analytically and numerically using the Path Integration approach. [Copyright &y& Elsevier]

In this paper, nonlinear dynamic characteristics and optimal control of giant magnetostrictive laminated plate (GMLP) subjected to in-plane stochastic excitation were studied. Von del Pol nonlinear item was introduced to interpret the hysteresis phenomenon of the strain-magnetic field intensity curve of giant magnetostrictive material, and the nonlinear dynamic model of GMLP subjected to in-plane stochastic excitation was developed. Local and global stochastic stabilities were analyzed according to largest Lyapunov exponent theory and singular boundary theory. The functions of steady-state probability density and joint probability density were obtained, and the condition of stochastic Hopf bifurcation was analyzed. The reliability function was solved from backward Kolmogorov equation, and the probability density of the first-passage time was obtained. Finally, the optimal control strategy was proposed in stochastic dynamic programming method. Numerical simulation shows that the stability of the solution varies with parameter, and stochastic Hopf bifurcation appears in the process; the reliability of the system was improved by optimal control, and the first-passage time was delayed. The result is helpful to engineering applications of GMLP. [ABSTRACT FROM AUTHOR]

One key idea behind morphological computation is that many difficulties of a control problem can be absorbed by the morphology of a robot. The performance of the controlled system naturally depends on the control architecture and on the morphology of the robot. Because of this strong coupling, most of the impressive applications in morphological computation typically apply minimalistic control architectures. Ideally, adapting the morphology of the plant and optimizing the control law interact so that finally, optimal physical properties of the system and optimal control laws emerge. As a first step toward this vision, we apply optimal control methods for investigating the power of morphological computation. We use a probabilistic optimal control method to acquire control laws, given the current morphology. We show that by changing the morphology of our robot, control problems can be simplified, resulting in optimal controllers with reduced complexity and higher performance. This concept is evaluated on a compliant four-link model of a humanoid robot, which has to keep balance in the presence of external pushes. [ABSTRACT FROM AUTHOR]

A minimax stochastic optimal semi-active control strategy for stochastically excited quasi-integrable Hamiltonian systems with parametric uncertainty by using magneto-rheological (MR) dampers is proposed. Firstly, the control problem is formulated as an n-degree-of-freedom (DOF) controlled, uncertain quasi-integrable Hamiltonian system and the control forces produced by MR dampers are split into the passive part and the semi-active part. Then the passive part is incorporated into the uncontrolled system. After that, the stochastic optimal semi-active control problem is solved by applying the minimax stochastic optimal control strategy based on the stochastic averaging method and stochastic differential game. The worst-case disturbances and the optimal controls are obtained by the minimax dynamical programming equation with the constraints of disturbance bounds and MR damper dynamics. Finally, the system response and the control performance are evaluated by using Monte Carlo simulation. An example of a two-DOF system with coupling damping and parametric uncertainty under Gaussian white noise excitations is worked out in detail to illustrate the procedure and effectiveness of the proposed control strategy, which is also compared with the clipped linear-quadratic-Gaussian control strategy to show the advantages. [ABSTRACT FROM PUBLISHER]

Abstract: New advances of the probability density evolution method for nonlinear stochastic systems are presented. The principle of preservation of probability, as a fundamental law of stochastic systems, is firstly revisited. It provides a unified basis for the probability density evolution equations holding for different types of stochastic systems. By integrating the random event description of this principle into the uncoupled physical equation, the generalized density evolution equation (GDEE) is derived. Some new perspectives, including the property of independent evolution of partial probability density function and the paths of ensemble evolution and point evolution, are provided towards setting a solid foundation for the methods of partition of probability-assigned space and numerical discretization of the GDEE. On this basis, new advances and extensions are outlined in the aspects of numerical methods, an extension of the GDEE to generic stochastic systems and applications to fluctuation of nonlinear systems and stochastic optimal control of structures. Problems to be further explored are pointed out. [Copyright &y& Elsevier]

Stochastic optimal semi-active control for stay cable multi-mode vibration attenuation by using magneto-rheological (MR) damper is developed. The Bingham model for an MR damper is used. The force produced by an MR damper is split into passive and active parts. The passive part is combined with structural damping forces into effective damping forces. The partially averaged Itô stochastic differential equations for controlled modal energies are derived by applying the stochastic averaging method for quasi-integrable Hamiltonian systems. Then the dynamical programming equation for controlled modal energies with an index involving control force is established by applying the stochastic dynamical programming principle, and a stochastic optimal semi-active control law is obtained by solving the dynamical programming equation. For controlled modal energies with an index not involving control force, bang-bang control law is obtained without solving a dynamical programming equation. A comparison between the two control laws shows that the stochastic optimal semi-active control strategy is superior to the bang-bang control strategy in the sense of higher control effectiveness and efficiency and less chattering. [ABSTRACT FROM PUBLISHER]

Multiple-Cumulant Cost Density-Shaping (MCCDS) control is proposed for the case when the system is linear and the cost is quadratic. This linear optimal control results from the minimization of an analytic, convex, non-negative function of cost cumulants and target cost cumulants. The MCCDS control allows the designer to shape the initial cost density with respect to a target density approximated by target cost cumulants. A numerical experiment shows that MCCDS control compares favorably with competing control paradigms in terms of official performance measures for inter-story drifts and per-story accelerations used in the first-generation structure benchmark for seismically excited buildings. [ABSTRACT FROM AUTHOR]

This paper is concerned with the stochastic optimal control problem of jump diffusions. The relationship between stochastic maximum principle and dynamic programming principle is discussed. Without involving any derivatives of the value function, relations among the adjoint processes, the generalized Hamiltonian and the value function are investigated by employing the notions of semijets evoked in defining the viscosity solutions. Stochastic verification theorem is also given to verify whether a given admissible control is optimal. [ABSTRACT FROM AUTHOR]

In this paper we introduce a new kind of backward stochastic differential equations, called ergodic BSDEs, which arise naturally in the study of optimal ergodic control. We study the existence, uniqueness, and regularity of solution to ergodic BSDEs. Then we apply these results to the optimal ergodic control of a Banach valued stochastic state equation. We also establish the link between the ergodic BSDEs and the associated Hamilton-Jacobi-Bellman equation. Applications are given to the optimal ergodic control of stochastic partial differential equations. [ABSTRACT FROM AUTHOR]

Abstract: A new approach for finding an exact analytical solution to the modified Hamilton–Jacobi–Bellman equation is proposed. Together with the recently developed hybrid solution method, the proposed strategy allows to find a solution to a whole class of stochastic optimal control problems with bounded in magnitude control force. [Copyright &y& Elsevier]

Abstract: Integrated water resources management (IWRM) is recognized worldwide as the reference paradigm to meet society’s long-term needs for water resources while maintaining essential ecological services and economic benefits. In previous publications [A. Castelletti, R. Soncini-Sessa, A procedural approach to strengthening integration and participation in water resource planning, Environmental Modelling & Software 21(10) (2006) 1455–1470; A. Castelletti, F. Pianosi, R. Soncini-Sessa, Integration, participation and optimal control in water resources planning and management, Applied Mathematics and Computation, (2007), doi:10.1016/j.amc.2007.09.069], the authors have already insisted on the need for a procedural approach to make the IWRM paradigm truly operational; they have emphasized the role played by dynamic optimization in rationalizing and facilitating the selection by the decision maker of a best compromise planning alternative. When planning alternatives also include management policies, as in the case of the water reservoir networks considered in this paper, the best compromise off-line policy resulting from the planning exercise has to be actually implemented in the daily management of the system. Here, again, dynamic optimization may play a central role, as it can be adopted on-line to improve the performance of the off-line policy by exploiting any new useful information available in real-time (e.g., inflow predictions, a power station being temporarily out of service, etc.). In this paper, this approach is explored through a real-world case study of a simple reservoir system. The off-line management policy computed in a previous planning process is refined on-line with a receding horizon control scheme combined with an inflow predictor. The results yield indications that the approach can provide significant advantages to cope with extreme events, particularly those occurring in unusual periods of the year. [Copyright &y& Elsevier]

Abstract: A method is presented for solving the infinite time Hamilton–Jacobi–Bellman (HJB) equation for certain state-constrained stochastic problems. The HJB equation is reformulated as an eigenvalue problem, such that the principal eigenvalue corresponds to the expected cost per unit time, and the corresponding eigenfunction gives the value function (up to an additive constant) for the optimal control policy. The eigenvalue problem is linear and hence there are fast numerical methods available for finding the solution. [Copyright &y& Elsevier]

We consider stochastic optimal control problems in Banach spaces. These problems are related to nonlinear controlled equations with dissipative nonlinearity and are treated via the backward stochastic differential equation approach, which also allows us to solve, in a mild sense, Hamilton-Jacobi-Bellman equations in Banach spaces. We apply the results to controlled stochastic heat and wave equations with a cost functional that is well defined on continuous functions, and to delay equations in spaces of p-integrable functions. [ABSTRACT FROM AUTHOR]