Search

We study the optimal consumption, investment and lifeinsurance purchase and selection strategies for a wage-earner with an uncertain lifetime with access to a financial market comprised of one risk-free security and one risky-asset whose prices evolve according to linear diffusions modulated by a continuous-time stochastic process determined by an additional diffusive nonlinear stochastic differential equation. The process modulating the linear diffusions may be regarded as an indicator describing the state of the economy in a given instant of time. Additionally, we allow the Brownian motions driving each of these equations to be correlated. The lifeinsurance market under consideration herein consists of a fixed number of providers offering pairwise distinct contracts. We use dynamic programming techniques to characterize the solutions to the problem described above for a general family of utility functions, studying the case of discounted constant relative risk aversion utilities with more detail. [ABSTRACT FROM AUTHOR]

We study an exit contract design problem, where one provides a universal exit contract to multiple heterogeneous agents, with which each agent chooses an optimal (exit) stopping time. The problem consists in optimizing the universal exit contract w.r.t. some criterion depending on the contract as well as the agents' exit times. Under a technical monotonicity condition, and by using Bank-El Karoui's representation of stochastic processes, we are able to transform the initial contract optimization problem into an optimal control problem. The latter is also equivalent to an optimal multiple stopping problem and the existence of the optimal contract is proved. We next show that the problem in the continuous-time setting can be approximated by a sequence of discrete-time ones, which would induce a natural numerical approximation method. We finally discuss the optimization problem over the class of all Markovian and/or continuous exit contracts. [ABSTRACT FROM AUTHOR]

We present a pair of adjoint optimal control problems characterizing a class of time-symmetric stochastic processes defined on random time intervals. The associated PDEs are of free-boundary type. The particularity of our approach is that it involves two adjoint optimal stopping times adapted to a pair of filtrations, the traditional increasing one and another, decreasing. They are the keys of the time symmetry of the construction, which can be regarded as a generalization of 'Schrödinger's problem' (1931–1932) to space-time domains. The relation with the notion of 'Hidden diffusions' is also described. [ABSTRACT FROM AUTHOR]

A controlled discrete-time stochastic process is proposed as a model for the state of a machine. In the proposed model, normal and random wear of the machine are considered. The random wear of the machine during a time unit can be either a discrete or a continuous random variable. The objective is to find a control policy that maximises the profits generated by the machine over its useful lifetime. In this problem, the optimiser must decide whether to do or not to do the maintenance work on the machine at each time unit. The significance of the paper is that the final time in the optimal control problem is a random variable denoting the first time the machine must be replaced. To obtain the optimal control, one can try to solve the dynamic programming equation, which reduces to a difference or an integral equation, satisfied by the value function. Finally, particular cases are solved exactly, or approximately, and explicitly to demonstrate and validate the proposed model. [ABSTRACT FROM AUTHOR]

This article studies the formation mission design problem for commercial aircraft in the presence of uncertainties. Specifically, it considers uncertainties in the departure times of the aircraft and in the fuel burn savings for the trailing aircraft. Given several commercial flights, the problem consists in arranging them in formation or solo flights and finding the trajectories that minimize the expected value of the DOC of the flights. The formation mission design problem is formulated as an optimal control problem of a stochastic switched dynamical system and solved using nonintrusive gPC-based stochastic collocation. The stochastic collocation method converts the SSOCP into an augmented deterministic switched optimal control problem. With this approach, a small number of sample points of the random parameters are used to jointly solve particular instances of the switched optimal control problem. The obtained solutions are then expressed as orthogonal polynomial expansions in terms of the random parameters using these sample points. This technique allows statistical and global sensitivity analysis of the stochastic solutions to be conducted at a low computational cost. The aim of this article is to establish if, in the presence of uncertainties, a formation mission is beneficial with respect to solo flight in terms of the expected value of the direct operating costs. Several numerical experiments have been conducted in which uncertainties on the departure times and on the fuel saving during formation flight have been considered. The obtained results demonstrate that benefits can be achieved even in the presence of these uncertainties. [ABSTRACT FROM AUTHOR]

Stochastic flows of an advective-diffusive nature are ubiquitous in biology and the physical sciences. Of particular interest is the problem of reconciling observed marginal distributions with a given prior posed by Schrödinger in 1932 and known as the Schrödinger Bridge Problem (SBP). It turns out that Schrödinger's problem can be viewed as both a modeling problem and a control problem. Due to their fundamental significance, the SBP and its deterministic (zero-noise limit) counterpart of Optimal Mass Transport (OMT) have recently received interest within a broad spectrum of disciplines, including physics, stochastic control, computer science, probability theory, and geometry. Yet, while the mathematics and applications of the SBP/OMT have been developing at a considerable pace, accounting for marginals of unequal mass has received scant attention; the problem of interpolating between "unbalanced" marginals has been approached by introducing source/sink terms into the transport equations, in an ad hoc manner, chiefly driven by applications in image registration. Nevertheless, losses are inherent in many physical processes, and thereby models that account for lossy transport may also need to be reconciled with observed marginals following Schrödinger's dictum; that is, it is necessary to adjust the probability of trajectories of particles, including those that do not make it to the terminal observation point, so that the updated law represents the most likely way that particles may have been transported, or have vanished, at some intermediate point. Thus, the purpose of this work is to develop such a natural generalization of the SBP for stochastic evolution with losses, whereupon particles are "killed" (jump into a coffin/extinction state) according to a probabilistic law, and thereby mass is gradually lost along their stochastically driven flow. Through a suitable embedding we turn the problem into an SBP for stochastic processes that combine diffusive and jump characteristics. Then, following a large-deviations formalism in the style of Schrödinger, given a prior law that allows for losses, we ask for the most probable evolution of particles along with the most likely killing rate as the particles transition between the specified marginals. Our approach differs sharply from previous work involving a Feynman-Kac multiplicative reweighing of the reference measure: The latter, as we argue, is far from Schrödinger's quest. An iterative scheme, generalizing the celebrated Fortet-IPF-Sinkhorn algorithm, permits us to compute the new drift and the new killing rate of the path-space solution measure. We finally formulate and solve a related fluid-dynamic control problem for the flow of one-time marginals where both the drift and the new killing rate play the role of control variables. [ABSTRACT FROM AUTHOR]

Based on a generative model (GM) and beliefs over hidden states, the free energy principle (FEP) enables an agent to sense and act by minimizing a free energy bound on Bayesian surprise, i.e., the negative logarithm of the marginal likelihood. Inclusion of desired states in the form of prior beliefs in the GM leads to active inference (ActInf). In this work, we aim to reveal connections between ActInf and stochastic optimal control. We reveal that, in contrast to standard cost and constraint-based solutions, ActInf gives rise to a minimization problem that includes both an information-theoretic surprise term and a model-predictive control cost term. We further show under which conditions both methodologies yield the same solution for estimation and control. For a case with linear Gaussian dynamics and a quadratic cost, we illustrate the performance of ActInf under varying system parameters and compare to classical solutions for estimation and control. [ABSTRACT FROM AUTHOR]

This paper presents a stochastic framework for the optimization of microgrids that has the functionality of providing flexibility services to System Operators (SOs) considering uncertainties in the energy forecast. The methodology is developed with the aim of being applied to complex microgrids composed of different distributed energy resources and hybrid energy storage systems (ESS). The associated optimization problem is operated in two stages: the first one performs a stochastic optimization of the microgrid in order to reserve an up/down regulation capacity with which to deal with the energy forecast uncertainties of the microgrid. The different microgrid devices are optimized by considering their operational costs in order to achieve their optimal operation in the Day-Ahead Market (DM). The second stage is used to re-schedule the initial planning according to the signal request and an economic offer from the SO. The control problem is developed using Stochastic Model Predictive Control (SMPC) techniques and Mixed-Integer Quadratic Programming (MIQP), owing to the presence of logic, integer, mixed and probabilistic variables. The simulation results show that the proposed methodology reduces the risk of undergoing up/down-penalty deviations in the Regulation Service Market (RM), also being able to provide flexibility services to the SOs, despite being subject to uncertainties in the energy forecast carried out for the microgrid. [ABSTRACT FROM AUTHOR]

We study the optimal control problem for a descriptor system whose evolution is described by Ito's differential-algebraic equation. The quadratic cost functional is considered. The main constraint is that the characteristic matrix pencil corresponding to the equation is regular. We establish the conditions for the existence and uniqueness of the optimal control and the corresponding optimal state. The results are illustrated on an example of a descriptor system that describes transient states in a radio engineering filter with random perturbations in the form of white noise. [ABSTRACT FROM AUTHOR]

Control based on vision data is a growing field of research and it is widespread in industry. The amount of data in each image and the processing needed to obtain control-relevant information from this data lead to significant delays with a large variability in the control loops. This often causes performance deterioration since in many cases the delay variability is not explicitly addressed in the control design. In this paper, we approach this problem by applying the ideas of recently developed model-based control design methods, which are tailored to address stochastic delays directly, to the motion control of an omnidirectional robot with a vision-based self-localization algorithm. The completion time or delay of the Random Sample Consensus (RANSAC) based localization algorithm is identified as a stochastic random variable with significant variability, illustrating the practical difficulties with data processing. Our main aim is to show that the novel deadline-driven and event-driven control designs significantly outperform a traditional periodic control implementation for a stochastic optimal control performance index. [ABSTRACT FROM AUTHOR]

As the application of probabilistic models in robotic applications increases, a systematic robot control approach considering the effects of uncertainty becomes indispensable. Inspired by human sensorimotor findings, in this paper, we study the stochastic optimal control problem with high-order cost statistics in order to synthesize uncertainty-dependent actions in robotic scenarios with multiple uncertainty sources. We present locally optimal risk-sensitive and cost-cumulant solutions for settings with nonlinear dynamics, multiple additive uncertainty sources, and nonquadratic costs. The influence of each uncertainty source on the cost can be individually parameterized offering additional flexibility in the control design. We further analyze the case in which the static uncertain parameters are involved. The simulations of several linear and nonlinear settings with nonquadratic costs and an experiment on a real robotic platform validate our approach and illustrate its peculiarities. [ABSTRACT FROM AUTHOR]

Stochastic optimal semi-active control for stay cable multi-mode vibration attenuation by using magneto-rheological (MR) damper is developed. The Bingham model for an MR damper is used. The force produced by an MR damper is split into passive and active parts. The passive part is combined with structural damping forces into effective damping forces. The partially averaged Itô stochastic differential equations for controlled modal energies are derived by applying the stochastic averaging method for quasi-integrable Hamiltonian systems. Then the dynamical programming equation for controlled modal energies with an index involving control force is established by applying the stochastic dynamical programming principle, and a stochastic optimal semi-active control law is obtained by solving the dynamical programming equation. For controlled modal energies with an index not involving control force, bang-bang control law is obtained without solving a dynamical programming equation. A comparison between the two control laws shows that the stochastic optimal semi-active control strategy is superior to the bang-bang control strategy in the sense of higher control effectiveness and efficiency and less chattering. [ABSTRACT FROM PUBLISHER]

Abstract: A new approach for finding an exact analytical solution to the modified Hamilton–Jacobi–Bellman equation is proposed. Together with the recently developed hybrid solution method, the proposed strategy allows to find a solution to a whole class of stochastic optimal control problems with bounded in magnitude control force. [Copyright &y& Elsevier]

Abstract: As a main step in the numerical solution of control problems in continuous time, the controlled process is approximated by sequences of controlled Markov chains, thus discretising time and space. A new feature in this context is to allow for delay in the dynamics. The existence of an optimal strategy with respect to the cost functional can be guaranteed in the class of relaxed controls. Weak convergence of the approximating extended Markov chains to the original process together with convergence of the associated optimal strategies is established. [Copyright &y& Elsevier]

We construct the explicit connection existing between a solvable model of the discrete velocities non-linear Boltzmann equation and the Hamilton–Bellman–Jacobi equation associated with a simple optimal control of a piecewise deterministic process. This study extends the known relation that exists between the Burgers equation and a simple controlled diffusion problem. In both cases the resulting partial differential equations can be linearized via a logarithmic transformation and hence offer the possibility to solve physically relevant non-linear field models in full generality. [ABSTRACT FROM AUTHOR]

We provide a natural derivation and interpretation for the uncertainty principle in quantum mechanics from the stochastic optimal control approach. We show that, in particular, the stochastic approach to quantum mechanics allows one to understand the uncertainty principle through the "thermodynamic equilibrium". A stochastic process with a gradient structure is key in terms of understanding the uncertainty principle and such a framework comes naturally from the stochastic optimal control approach to quantum mechanics. The symmetry of the system is manifested in certain non-vanishing and invariant covariances between the four-position and the four-momentum. In terms of interpretation, the results allow one to understand the uncertainty principle through the lens of scientific realism, in accordance with empirical evidence, contesting the original interpretation given by Heisenberg. [ABSTRACT FROM AUTHOR]