Search

This paper is concerned with a class of coupled forward–backward stochastic differential equations (FBSDEs, for short) involving time delays and time advancements on infinite horizon. By introducing a randomized Lipschitz condition and a randomized monotonicity condition, the unique solvability of FBSDEs is obtained. Then the theoretical result is applied to a linear–quadratic (LQ, for short) problem of a time-delayed system with random coefficients. An explicit expression of the unique optimal control is obtained. [ABSTRACT FROM AUTHOR]

The solution of a stochastic optimal control problem may be associated with that of the Hamilton–Jacobi–Bellman (HJB) equation, which is a second order partial differential equation subject to a terminal condition. When this equation is semilinear and satisfies certain other constraints, it can be solved via a nonlinear version of the Feynman–Kac formula. According to this approach, the solution to the HJB equation can be obtained by simulating an associated pair of partly coupled forward–backward stochastic differential equations. Although an elegant way to interpret and solve a partial differential equation, simulating the system of forward–backward equations can be computationally inefficient. In this work, the HJB equation pertaining to the optimal control problem is reformulated such that instead of the given terminal condition, it is now subject to an appropriate initial condition. In the process, while the total cost associated with the control problem remains unchanged, pathwise solutions may not. Associated with the new partial differential equation, we then derive a set of stochastic differential equations whose solutions move only forward in time. This approach has a significant computational advantage over the original formulation. Moreover, since the forward–backward approach generally requires simulating stochastic differential equations over the current to the terminal time at every step, the integration errors may accumulate and carry forward in estimating the control. This error is particularly high initially since the time of integration is the longest there. The proposed method, numerically implemented for the control of stochastically excited oscillators, is free of such errors and hence more robust. Unsurprisingly, it also exhibits lower sampling variance. [Display omitted] • A Monte Carlo scheme with forward time integration for stochastic optimal control. • Based on a modified HJB equation, now supplied with an initial condition. • Achieved through a smearing of the terminal cost by Ito's formula. • Vastly reduced computational cost and sampling variance. • Advantages illustrated with numerical examples. [ABSTRACT FROM AUTHOR]

• Reliability-based stochastic optimal control method via DPIM of building is proposed. • The method is effective for vibration reduction of frame under near-fault ground motions. • The control scheme achieves a proper balance between control efficacy and control force. • Velocity pulses of near-fault motions affect greatly control scheme of frame building. Most of stochastic optimal control methods were developed on the basis of Itô stochastic differential equation, which assumes that the external excitation is white Gaussian noise or filtered white Gaussian noise. However, this assumption is far from the real excitations, which hinders the applications of stochastic optimal control theory. In this paper, the reliability-based stochastic optimal control via direct probability integral method (DPIM) for building structure is proposed, which is applicable to performance-based design of general control systems of linear structures under non-stationary and non-white random excitations. Firstly, the DPIM is utilized to accurately and efficiently compute dynamic reliability of the controlled system. Then, the reliability-based objective function is suggested to optimize the parameters of the placed control devices of building structure, and reliability-based maximum story controllability index is established to determine the optimal placement of control devices, thus fulfilling the stochastic optimal control of structure. Finally, a 10-story shear frame building controlled by active tendons under random excitations of near-fault earthquake ground motions verifies the effectiveness of the proposed optimal control method. Moreover, the optimal control schemes of frame structure under near-fault non-pulse and impulsive ground motions are compared, indicating that the velocity pulses of near-fault ground motions impose significant effect on structural responses, and the more control energy is needed for guaranteeing structural safety. [ABSTRACT FROM AUTHOR]

In stochastic optimal control (SOC) one minimizes the average cost-to-go, that consists of the cost-of-control (amount of efforts), cost-of-space (where one wants the system to be) and the target cost (where one wants the system to arrive), for a system participating in forced and controlled Langevin dynamics. We extend the SOC problem by introducing an additional cost-of-dynamics, characterized by a vector potential. We propose derivation of the generalized gauge-invariant Hamilton-Jacobi-Bellman equation as a variation over density and current, suggest hydrodynamic interpretation and discuss examples, e.g., ergodic control of a particle-within-a-circle, illustrating non-equilibrium spacetime complexity. [ABSTRACT FROM AUTHOR]

In this paper, the infinite horizon optimal control problems for singular diffusion processes are considered from the viewpoints of Markov decision processes and perturbation analysis, where the singularity of diffusion means that the covariance matrix of the system noise is allowed to be degenerate. A formula of performance difference under two different controls is derived and leads to a comparison theorem. By the comparison theorem, starting from a control, a so-called better control can be selected. Therefore, a control policy iteration algorithm is developed, by which the performance improves step by step and converges to the optimal one. When this applies to the stochastic affine nonlinear regulator and stochastic linear quadratic optimal control problems, better control can be constructed in a closed form. It is also shown that when the considered stochastic systems degenerate to the deterministic ones, the proposed algorithm reduces to the adaptive dynamic programming algorithm [J. J. Murray, C. J. Cox, G. G. Lendaris, and R. Saeks, Adaptive dynamic programming, IEEE Trans. Systems Man Cybernet., 32 (2002), pp. 140-153] for the affine nonlinear systems and to the well-known Kleinman algorithm [D. L. Kleinman, On an iterative technique for Riccati equation computation, IEEE Trans. Automat. Control, 13 (1968), pp. 114-115] for the linear quadratic optimal control problem. [ABSTRACT FROM AUTHOR]

This paper studies large population dynamic games involving nonlinear stochastic dynamical systems with agents of the following mixed types: (i) a major agent and (ii) a population of N minor agents where N is very large. The major and minor agents are coupled via both (i) their individual nonlinear stochastic dynamics and (ii) their individual finite time horizon nonlinear cost functions. This problem is analyzed by the so-called-Nash mean field game theory. A distinct feature of the mixed agent mean field game problem is that even asymptotically (as the population size approaches in?nity) the noise process of the major agent causes random fluctuation of the N mean field behavior of the minor agents. To deal with this, the overall asymptotic (N →∞)mean field game problem is decomposed into (i) two nonstandard stochastic optimal control problems with random coefficient processes which yield forward adapted stochastic best response control processes determined from the solution of (backward in time) stochastic Hamilton-Jacobi-Bellman (SHJB) equations and (ii) two stochastic coefficient McKean-Vlasov (SMV) equations which characterize the state of the major agent and the measure determining the mean field behavior of the minor agents. This yields a stochastic mean field game (SMFG) system which is in contrast to the deterministic mean field game systems of standard MFG problems with only minor agents. Existence and uniqueness of the solutions to SMFG systems (SHJB and SMV equations) is established by a fixed point argument in the Wasserstein space of random probability measures. In the case where minor agents are coupled to the major agent only through their cost functions, the ∈N-Nash equilibrium property of the SMFG best responses is shown for a finite N population system where ∈N = O(1/√N). [ABSTRACT FROM AUTHOR]

We obtain a maximum principle for stochastic control problem of general controlled stochastic differential systems driven by fractional Brownian motions (of Hurst parameter H>1/2). This maximum principle specifies a system of equations that the optimal control must satisfy (necessary condition for the optimal control). This system of equations consists of a backward stochastic differential equation driven by both fractional Brownian motions and the corresponding underlying standard Brownian motions. In addition to this backward equation, the maximum principle also involves the Malliavin derivatives. Our approach is to use conditioning and Malliavin calculus. To arrive at our maximum principle we need to develop some new results of stochastic analysis of the controlled systems driven by fractional Brownian motions via fractional calculus. Our approach of conditioning and Malliavin calculus is also applied to classical system driven by standard Brownian motions while the controller has only partial information. As a straightforward consequence, the classical maximum principle is also deduced in this more natural and simpler way. [ABSTRACT FROM AUTHOR]

Research insurance funds investment based on constant elasticity of variance (CEV) model, consider a model which the risky asset is modeled by CEV model and the aggregate claims are inodeled by a Brownian motion with drift. As employment of premium is different from ordinary, which means that the insurer should keep an eye on underwrite risk when he use insurance funds, assume that investment risk has a linear correlation with underwrite risk. According to stochastic control theory, derive the HJB equation related with insurance problem. This equation is non-linear partial differential equation, yet it is difficult to solve it, change primary problem to the dual problem by using Legendre transform. Through setting the parameter values, the optimal investment strategy for an insurer with CARA or CRRA utility flmction is presented and the relevant analysis is given, which provides important practical significance for an insurer to invest. [ABSTRACT FROM AUTHOR]

For a sequence of dynamic optimization problems, we aim at discussing a notion of consistency over time. This notion can be informally introduced as follows. At the very first time step t, the decision maker formulates an optimization problem that yields optimal decision rules for all the forthcoming time steps t, t,..., T; at the next time step t, he is able to formulate a new optimization problem starting at time t that yields a new sequence of optimal decision rules. This process can be continued until the final time T is reached. A family of optimization problems formulated in this way is said to be dynamically consistent if the optimal strategies obtained when solving the original problem remain optimal for all subsequent problems. The notion of dynamic consistency, well-known in the field of economics, has been recently introduced in the context of risk measures, notably by Artzner et al. (Ann. Oper. Res. 152(1):5-22, ) and studied in the stochastic programming framework by Shapiro (Oper. Res. Lett. 37(3):143-147, ) and for Markov Decision Processes (MDP) by Ruszczynski (Math. Program. 125(2):235-261, ). We here link this notion with the concept of 'state variable' in MDP, and show that a significant class of dynamic optimization problems are dynamically consistent, provided that an adequate state variable is chosen. [ABSTRACT FROM AUTHOR]

In this work we consider a stochastic optimal control problem with either convex control constraints or finitely many equality and inequality constraints over the final state. Using the variational approach, we are able to obtain first and second order expansions for the state and cost function, around a local minimum. This fact allows us to prove general first order necessary condition and, under a geometrical assumption over the constraint set, second order necessary conditions are also established. We end by giving second order optimality conditions for problems with constraints on expectations of the final state. [ABSTRACT FROM AUTHOR]

Abstract: This paper is concerned with one kind of forward–backward linear quadratic stochastic control problem whose system is described by a linear anticipated forward–backward stochastic differential delayed equation. The explicit form of the optimal control is derived. Optimal state feedback regulators are studied in two special cases. For the case with delay in just the control variable, the optimal state feedback regulator is obtained by the Riccati equation. For the other case with delay in just the state variable, the optimal state feedback regulator is analyzed by the value function approach. [Copyright &y& Elsevier]

For discrete-time linear-quadratic Gaussian (LQG) control problems, a utility function on the expectation and the variance of the conventional performance index is considered. The utility function is viewed as an overall objective of the system and can perform the optimal trade-off between the mean and the variance of performance index. The nonlinear utility function is first converted into an auxiliary parameters optimisation problem about the expectation and the variance. Then an optimal closed-loop feedback controller for the nonseparable mean–variance minimisation problem is designed by nonlinear mathematical programming. Finally, simulation results are given to verify the algorithm's effectiveness obtained in this article. [ABSTRACT FROM AUTHOR]

Stochastic optimal semi-active control for stay cable multi-mode vibration attenuation by using magneto-rheological (MR) damper is developed. The Bingham model for an MR damper is used. The force produced by an MR damper is split into passive and active parts. The passive part is combined with structural damping forces into effective damping forces. The partially averaged Itô stochastic differential equations for controlled modal energies are derived by applying the stochastic averaging method for quasi-integrable Hamiltonian systems. Then the dynamical programming equation for controlled modal energies with an index involving control force is established by applying the stochastic dynamical programming principle, and a stochastic optimal semi-active control law is obtained by solving the dynamical programming equation. For controlled modal energies with an index not involving control force, bang-bang control law is obtained without solving a dynamical programming equation. A comparison between the two control laws shows that the stochastic optimal semi-active control strategy is superior to the bang-bang control strategy in the sense of higher control effectiveness and efficiency and less chattering. [ABSTRACT FROM PUBLISHER]

A framework is introduced for the identification of controls for single-class time-varying queueing networks that are asymptotically optimal in the so-called uniform acceleration regime. A related, but simpler, first-order (or fluid) control problem is first formulated. For a class of performance measures that satisfy a certain continuity property, it is then shown that any sequence of policies whose performances approach the infimum in the fluid control problem is asymptotically optimal for the original network problem. Examples of performance measures with this property are described, and simulations implementing asymptotically optimal policies are presented. The use of directional derivatives of the reflection map for solving fluid control problems is also illustrated. This work serves to complement a large body of literature on asymptotically optimal controls for time-homogeneous networks. [ABSTRACT FROM AUTHOR]

In this article we consider the problem of stochastic optimal control in continuous-time and state-action space of systems with state constraints. These systems typically appear in the area of robotics, where hard obstacles constrain the state space of the robot. A common approach is to solve the problem locally using a linear-quadratic Gaussian (LQG) method. We take a different approach and apply path integral control as introduced by Kappen (Kappen, H.J. (2005a), 'Path Integrals and Symmetry Breaking for Optimal Control Theory', Journal of Statistical Mechanics: Theory and Experiment, 2005, P11011; Kappen, H.J. (2005b), 'Linear Theory for Control of Nonlinear Stochastic Systems', Physical Review Letters, 95, 200201). We use hybrid Monte Carlo sampling to infer the control. We introduce an adaptive time discretisation scheme for the simulation of the controlled dynamics. We demonstrate our approach on two examples, a simple particle in a halfspace and a more complex two-joint manipulator, and we show that in a high noise regime our approach outperforms the iterative LQG method. [ABSTRACT FROM AUTHOR]

Abstract: A new approach for finding an exact analytical solution to the modified Hamilton–Jacobi–Bellman equation is proposed. Together with the recently developed hybrid solution method, the proposed strategy allows to find a solution to a whole class of stochastic optimal control problems with bounded in magnitude control force. [Copyright &y& Elsevier]

A prime objective of modeling genetic regulatory networks is the identification of potential targets for therapeutic intervention. To date, optimal stochastic intervention has been studied in the context of probabilistic Boolean networks, with the control policy based on the transition probability matrix of the associated Markov chain and dynamic programming used to find optimal control policies. Dynamical programming algorithms are problematic owing to their high computational complexity. Two additional computationally burdensome issues that arise are the potential for controlling the network and identifying the best gene for intervention. This paper proposes an algorithm based on mean first-passage time that assigns a stationary control policy for each gene candidate. It serves as an approximation to an optimal control policy and, owing to its reduced computational complexity, can be used to predict the best control gene. Once the best control gene is identified, one can derive an optimal policy or simply utilize the approximate policy for this gene when the network size precludes a direct application of dynamic programming algorithms. A salient point is that the proposed algorithm can be model-free. It can be directly designed from time-course data without having to infer the transition probability matrix of the network. [ABSTRACT FROM AUTHOR]

We tackle the problem of trajectory planning in an environment comprised of a set of obstacles with uncertain time-varying locations. The uncertainties are modeled using widely accepted Gaussian distributions, resulting in a chance-constrained program. Contrary to previous approaches however, we do not assume perfect knowledge of the moments of the distribution, and instead estimate them through finite samples available from either sensors or past data. We derive tight concentration bounds on the error of these estimates to sufficiently tighten the chance-constraint program. As such, we provide provable guarantees on satisfaction of the chance-constraints corresponding to the nominal yet unknown moments. We illustrate our results with two autonomous vehicle trajectory planning case studies. [ABSTRACT FROM AUTHOR]

• Steady-state policies of dynamic programming in long-term hydrothermal scheduling. • Assessment of stochastic dynamic programming in long-term hydrothermal scheduling. • Influence of system's hydro generation share in long-term hydrothermal scheduling. • Inflows for deterministic dynamic programming in long-term hydrothermal scheduling. • Policies for long-term hydrothermal scheduling of single-reservoir systems. Markov stochastic dynamic programming (MSDP) has been extensively used for long-term hydrothermal systems (LTHS) of single-reservoir systems and has inspired many approaches to multiple-reservoir systems, despite the lack of comprehensive studies that test their performance. This study is concerned with the evaluation of classic dynamic programming policies in LTHS of single-reservoir systems considering the operation in steady-state regime. The Markov stochastic dynamic programming was compared with deterministic dynamic programming (DDP) considering average inflows, and unconditional stochastic dynamic programming (USDP) considering uncorrelated inflows. Performance measures were obtained through simulation over long horizons using historical and synthetically generated inflows. Sensitivity analysis related to hydro plant characteristics, the hydro generation share in the system, and the inflow considered in DDP was performed. Results show that MSDP provides small performance gains whatever the hydro plant considered, that become smaller with the reduction in the share of hydro generation in the system, and the proper consideration of inflows in DDP. [ABSTRACT FROM AUTHOR]

For an optimal control problem of an Itô's type stochastic differential equation, the control process could be taken in open-loop or closed-loop forms. In the standard literature, provided appropriate regularity, the value functions under these two types of controls are equal and are the unique (viscosity) solution to the corresponding (path-dependent) HJB equation. In this short note, we provide a counterexample in the path dependent setting showing that these value functions can be different in general. [ABSTRACT FROM AUTHOR]

Abstract: As a main step in the numerical solution of control problems in continuous time, the controlled process is approximated by sequences of controlled Markov chains, thus discretising time and space. A new feature in this context is to allow for delay in the dynamics. The existence of an optimal strategy with respect to the cost functional can be guaranteed in the class of relaxed controls. Weak convergence of the approximating extended Markov chains to the original process together with convergence of the associated optimal strategies is established. [Copyright &y& Elsevier]

We construct the explicit connection existing between a solvable model of the discrete velocities non-linear Boltzmann equation and the Hamilton–Bellman–Jacobi equation associated with a simple optimal control of a piecewise deterministic process. This study extends the known relation that exists between the Burgers equation and a simple controlled diffusion problem. In both cases the resulting partial differential equations can be linearized via a logarithmic transformation and hence offer the possibility to solve physically relevant non-linear field models in full generality. [ABSTRACT FROM AUTHOR]

A strategy for nonlinear stochastic optimal control of partially observable linear structures is proposed and illustrated with linear building structures equipped with control devices and sensors under horizontal ground acceleration excitation. The control problem of a partially observable structure is first converted into that of a completely observable structure based on the separation principle. Then, a partially averaged control system of Itoˆ equations is obtained from the completely observable structure by using the stochastic averaging method for quasi-Hamiltonian systems. Dynamical programming equations for finite and infinite time-interval controls are established based on the stochastic dynamical programming principle and solved to obtain the optimal control law and value function. Finally, the response of controlled structure is obtained from solving the Fokker–Planck–Kolmogorov equation associated with the fully averaged system of Itoˆ equations. The numerical results for a five-story building structure model are obtained by using the proposed control strategy and compared with those by using linear quadratic Gaussian control strategy to show the effectiveness and efficiency of the proposed strategy. [Copyright &y& Elsevier]

We introduce a notion of pathwise optimality for stochastic control problems over an infinite time horizon, and give sufficient conditions for the existence of pathwise optimal controls. We analyze both diffusion processes and processes with discrete state space. [ABSTRACT FROM AUTHOR]

We establish a generalization of the Noether theorem for stochastic optimal control problems. Exploiting the tools of jet bundles and contact geometry, we prove that from any (contact) symmetry of the Hamilton–Jacobi–Bellman equation associated with an optimal control problem it is possible to build a related local martingale. Moreover, we provide an application of the theoretical results to Merton's optimal portfolio problem, showing that this model admits infinitely many conserved quantities in the form of local martingales. [ABSTRACT FROM AUTHOR]

In this paper, an open problem is solved, for the stochastic optimal control problem with delay where the control domain is nonconvex and the diffusion term contains both control and its delayed term. Inspired by previous results about delayed stochastic control systems, Peng's global stochastic maximum principle is generalized to the time delayed case. A special backward stochastic differential equation is introduced to deal with the cross terms, when applying the duality technique. Comparing with the classical result, the maximum condition contains an indicator function, which in fact is the characteristic of the stochastic optimal control problem with delay. Furthermore, to illustrate the applications of our theoretical results, three dynamic optimization problems are addressed based on the global maximum principle. [ABSTRACT FROM AUTHOR]

In this letter we provide several informative tight error bounds when using value function approximators for the risk-sensitive cost setting for a given policy represented using exponential utility. The novelty of our approach is that we make use of the irreducibility of the underlying Markov chain (resulting in better bounds using Perron–Frobenius eigenvectors) to derive new bounds whereas the earlier work used primarily the spectral variation bound which holds for any matrix, hence did not make use of the irreducibility. All our bounds have a perturbation term for large state spaces. We also present examples where we show that the new bounds perform 90-100% better than the earlier proposed spectral variation bound. [ABSTRACT FROM AUTHOR]

This paper investigates optimal power management of a fuel cell hybrid small unmanned aerial vehicle (sUAV) from the perspective of endurance (time of flight) maximization in a stochastic environment. Stochastic drift counteraction optimal control is exploited to obtain an optimal policy for power management that coordinates the operation of the fuel cell and battery to maximize the expected flight time while accounting for the limits on the rate of change of fuel cell power output and the orientation dependence of fuel cell efficiency. The proposed power management strategy accounts for known statistics in transitions of propeller power and climb angle during the mission, but does not require the exact preview of their time histories. The optimal control policy is generated offline using value iterations implemented in Cython, demonstrating an order of magnitude speedup as compared to MATLAB. It is also shown that the value iterations can be further sped up using a discount factor, but at the cost of decreased performance. Simulation results for a 1.5 kg sUAV are reported that illustrate the optimal coordination between the fuel cell and the battery during aircraft maneuvers, including a turnpike in the battery state of charge ( S O C ) trajectory. As the fuel cell is not able to support fast changes in power output, the optimal policy is shown to charge the battery to the turnpike value if starting from a low initial S O C value. If starting from a high S O C value, the battery energy is used till a turnpike value of the S O C is reached with further discharge delayed to later in the flight. For the specific scenarios and simulated sUAV parameters considered, the results indicate the capability of up to 2.7 h of flight time. [ABSTRACT FROM AUTHOR]

We provide a rigorous mathematical formulation of Deep Learning (DL) methodologies through an in-depth analysis of the learning procedures characterizing Neural Network (NN) models within the theoretical frameworks of Stochastic Optimal Control (SOC) and Mean-Field Games (MFGs). In particular, we show how the supervised learning approach can be translated in terms of a (stochastic) mean-field optimal control problem by applying the Hamilton–Jacobi–Bellman (HJB) approach and the mean-field Pontryagin maximum principle. Our contribution sheds new light on a possible theoretical connection between mean-field problems and DL, melting heterogeneous approaches and reporting the state-of-the-art within such fields to show how the latter different perspectives can be indeed fruitfully unified. [ABSTRACT FROM AUTHOR]

This research article shows how the pricing of derivative securities can be seen from the context of stochastic optimal control theory and information theory. The financial market is seen as an information processing system, which optimizes an information functional. An optimization problem is constructed, for which the linearized Hamilton–Jacobi–Bellman equation is the Black–Scholes pricing equation for financial derivatives. The model suggests that one can define a reasonable Hamiltonian for the financial market, which results in an optimal transport equation for the market drift. It is shown that in such a framework, which supports Black–Scholes pricing, the market drift obeys a backwards Burgers equation and that the market reaches a thermodynamical equilibrium, which minimizes the free energy and maximizes entropy. [ABSTRACT FROM AUTHOR]

In this paper, we consider the problem of how to run an insurance company to minimize the ruin probability in a finite time horizon. The value function v with the corresponding optimal reinsure strategy a ∗ is the solution of the following fully nonlinear equation v t − 1 2 σ 0 2 v x x + γ v x − inf 0 ≤ a ≤ 1 ( 1 2 σ 2 a 2 v x x + μ a v x ) = 0 under initial–boundary condition v (0 , t) = 1 , v (x , 0) = 0. The main effort is to analyze the properties of the solution and the free boundary to show the optimal decision for the company. [ABSTRACT FROM AUTHOR]

We provide a natural derivation and interpretation for the uncertainty principle in quantum mechanics from the stochastic optimal control approach. We show that, in particular, the stochastic approach to quantum mechanics allows one to understand the uncertainty principle through the "thermodynamic equilibrium". A stochastic process with a gradient structure is key in terms of understanding the uncertainty principle and such a framework comes naturally from the stochastic optimal control approach to quantum mechanics. The symmetry of the system is manifested in certain non-vanishing and invariant covariances between the four-position and the four-momentum. In terms of interpretation, the results allow one to understand the uncertainty principle through the lens of scientific realism, in accordance with empirical evidence, contesting the original interpretation given by Heisenberg. [ABSTRACT FROM AUTHOR]

Optimal control is a prominent approach in robotics and movement neuroscience, among other fields of science. Methods for deriving optimal choices of action have been classically devised either in deterministic or stochastic settings. Here, we consider a setting in-between that retains the stochastic aspect of the controlled system but assumes deterministic open-loop control actions. The rationale stems from observations about the neural control of movement which highlighted that relatively stable behaviors can be achieved without feedback circuitry, via open-loop motor commands adequately tuning the mechanical impedance of the neuromusculoskeletal system. Yet, effective methods for deriving optimal open-loop controls for stochastic systems are lacking overall. This work presents a continuous-time approach based on statistical linearization techniques for the efficient computation of optimal open-loop controls for a broad class of stochastic optimal control problems. We first show that non-trivial departure from the optimal solutions of classical deterministic and stochastic approaches may arise for simple synthetic examples, thereby stressing the originality of the framework. We then exemplify its potential relevance to the planning of biological movement by showing that a well-known phenomenon in motor control, referred to as muscle co-contraction, occurs naturally. More generally, this stochastic optimal control framework may be suited to other fields where the design of optimal open-loop actions is relevant. [ABSTRACT FROM AUTHOR]

In this paper, we provide a closed-form solution to an optimal portfolio execution problem with stochastic price impact and stochastic net demand pressure. Specifically, each trade of an investor has temporary and permanent price impacts, both of which are driven by a continuous-time Markov chain; whereas the net demand pressure from other inventors is modelled by an Ornstein—Uhlenbeck process. The investor optimally liquidates his portfolio to maximize his expected revenue netting his cumulative inventory cost over a finite time. Such a problem is first reformulated as an optimal stochastic control problem for a Markov jump linear system. Then, we derive the value function and the optimal feedback execution strategy in terms of the solutions to coupled differential Riccati equations. Under some mild conditions, we prove that the coupled system is well-posed, and establish a verification theorem. Financially, our closed-form solution shows that the investor optimally liquidates his portfolio towards a dynamic benchmark. Moreover, the investor trades aggressively (conservatively) in the state of low (high) price impact. [ABSTRACT FROM AUTHOR]

With the penetration of renewable generation, electric vehicles and other random factors in power systems, the stochastic disturbances are increasing significantly, which are necessary to be handled for guarantying the security of systems. A novel stochastic optimal control strategy is proposed in this paper to reduce the impact of such stochastic continuous disturbances on power systems. The proposed method is effective in solving the problems caused by the stochastic continuous disturbances and has two significant advantages. First, a simplified and effective solution is proposed to analyze the system influenced by the stochastic disturbances. Second, a novel optimal control strategy is proposed in this paper to effectively reduce the impact of stochastic continuous disturbances. To be specific, a novel excitation controlled power systems model with stochastic disturbances is built in the quasi-generalized Hamiltonian form, which is further simplified into a lower-dimension model through the stochastic averaging method. Based on this Itô equation, a novel optimal control strategy to achieve the intra-region probability maximization is established for power systems by using the dynamic programming method. Finally, the intra-region probability increases in controlled systems, which confirms the effectiveness of the proposed control strategy. The proposed control method has advantages on controlling the fluctuation of system state variables within a desired region under the influence of stochastic disturbances, which means improving the security of stochastic systems. With more stochasticity in the future, the proposed control method based on the stochastic theory will play a novel way to relieve the impact of stochastic disturbances. [ABSTRACT FROM AUTHOR]

A conventional approach to solving stochastic optimal control problems with time-dependent uncertainties involves the use of the stochastic maximum principle (SMP) technique. For large-scale problems, however, such an algorithm frequently leads to convergence complexities when solving the two-point boundary value problem resulting from the optimality conditions. An alternative approach consists of using continuous random variables to capture uncertainty through sampling-based methods embedded within an optimization strategy for the decision variables; such a technique may also fail due to the computational intensity involved in excessive model calculations for evaluating the objective function and its derivatives for each sample. This paper presents a new approach to solving stochastic optimal control problems with time-dependent uncertainties based on BONUS (Better Optimization algorithm for Nonlinear Uncertain Systems). The BONUS has been used successfully for non-linear programming problems with static uncertainties, but we show here that its scope can be extended to the case of optimal control problems with time-dependent uncertainties. A batch reactor for biodiesel production was used as a case study to illustrate the proposed approach. Results for a maximum profit problem indicate that the optimal objective function and the optimal profiles were better than those obtained by the maximum principle. [ABSTRACT FROM AUTHOR]

Autonomous exploration of environmental fields is one of the most promising tasks to be performed by fleets of mobile underwater robots. The goal is to maximize the information gain during the exploration process by integrating an information-metric into the path-planning and control step. Therefore, the system maintains an internal belief representation of the environmental field which incorporates previously collected measurements from the real field. In contrast to surface robots, mobile underwater systems are forced to run all computations on-board due to the limited communication bandwidth in underwater domains. Thus, reducing the computational cost of field exploration algorithms constitutes a key challenge for in-field implementations on micro underwater robot teams. In this work, we present a computationally efficient exploration algorithm which utilizes field belief models based on Gaussian Processes, such as Gaussian Markov random fields or Kalman regression, to enable field estimation with constant computational cost over time. We extend the belief models by the use of weighted shape functions to directly incorporate spatially continuous field observations. The developed belief models function as information-theoretic value functions to enable path planning through stochastic optimal control with path integrals. We demonstrate the efficiency of our exploration algorithm in a series of simulations including the case of a stationary spatio-temporal field. [ABSTRACT FROM AUTHOR]

Mass vaccination campaigns play major roles in the war against epidemics. Such prevention strategies cannot always reach their goals significantly without the help of media and awareness campaigns used to prevent contacts between susceptible and infected people. Feelings of fear, infodemics, and misconception could lead to some fluctuations of such policies. In addition to the vaccination strategy, the movement restriction approach is essential because of the factor of mobility or travel. However, anti-epidemic border measures may also be disturbed if some infected travelers manage to escape and infiltrate into a safer region. In this paper, we aim to study infection dynamics related to the spatial spread of an epidemic in interconnected regions in the presence of random perturbations caused by the three above-mentioned reasons. Therefore, we devise a stochastic multi-region epidemic model in which contacts between susceptible and infected populations, vaccination-based and movement restriction optimal control approaches are all assumed to be unpredictable, and then, we discuss the effectiveness of such policies. In order to reach our goal, we employ a stochastic maximum principle version for noised systems, state and prove the sufficient and necessary conditions of optimality, and finally provide the numerical results obtained using a stochastic progressive-regressive schemes method. [ABSTRACT FROM AUTHOR]