Search

This paper considers an optimal control problem for a time-invariant linear stochastic system with discrete time, scalar unbounded control, additive noise, and a probabilistic criterion for retaining its trajectories in a given neighborhood of zero. We use dynamic programming and two-sided Bellman function estimates to derive analytical expressions for the optimal control at two time steps and a suboptimal control on any control horizon. The effectiveness of these controls is illustrated on a numerical example. [ABSTRACT FROM AUTHOR]

This paper considers an optimal control problem for a discrete-time stochastic system with the probability of first reaching the boundaries of a given domain as the optimality criterion. Dynamic programming-based sufficient conditions of optimality are formulated and proved. The isobells of levels 1 and 0 of the Bellman function are used for obtaining two-sided estimates of the right-hand side of the dynamic programming equation, two-sided estimates of the Bellman function, and two-sided estimates of the optimal-value function of the probabilistic criterion. A suboptimal control design method is proposed. The conditions of equivalence to an optimal control problem with a probabilistic terminal criterion are established. An illustrative example is given. [ABSTRACT FROM AUTHOR]

This paper examines an optimal control problem with a probabilistic criterion for retaining the trajectories of a discrete-time stochastic system in given sets. The dynamic programming method is employed for obtaining the isobells of levels 1 and 0 of the Bellman function, two-sided estimates for the right-hand side of the dynamic programming equation, two-sided estimates for the Bellman function, and the optimal-value function of the probabilistic criterion. These results are then used for deriving an approximate formula for the optimal control. As an illustrative example the problem of keeping an inverted pendulum in the neighborhood of an unstable equilibrium is considered. [ABSTRACT FROM AUTHOR]

In this paper, the optimal control problem for a discrete-time stochastic system with a general-form probabilistic criterion is considered. Using dynamic programming and the properties of the Bellman function, new two-sided bounds of this function that refine the earlier results are constructed. The derived bounds are then adopted to justify the application of the modified strategy that is optimal in the two-step investment portfolio management problem under risk to the corresponding multistep problem. An example that illustrates the advantages of such a strategy over other well-known strategies is given. [ABSTRACT FROM AUTHOR]

Consideration was given to the design of the optimal control of the general discrete stochastic system with a criterion as the probability of the state vector sojourn in the given sets at each time instant. Derived were relations of the dynamic programming enabling one to establish an optimal solution in the class of Markov strategies without extension of the state vector with subsequent reduction to an equivalent problem with the probabilistic terminal criterion. Consideration was given to the problem of one-parameter correction of the flying vehicle trajectory. An analytical solution was established. [ABSTRACT FROM AUTHOR]

In this article, we study and compare two approaches to solving stochastic optimal control problems with an expectation constraint on the final state. The case of a probability constraint is included in this framework. The first approach is based on a dynamic programming principle and the second one uses Lagrange relaxation. In this article, we focus on discrete-time problems, but the two discussed approaches can be applied to discretized continuous-time problems. [ABSTRACT FROM AUTHOR]

In this paper, we address the problem of optimal management of renewable resources such as agricultural commodities and fishery production. For that purpose, we consider the population associated with such commodities and assume that its size evolves according to a logistic growth model with a predation term given by a Holling type- n functional response. Additionally, we assume that such population is subject to random fluctuations, modeled by a diffusive term driven by a one-dimensional Brownian motion and having a power-type coefficient, thus endowing the model under consideration with the property of having constant elasticity of variance. Since the stochastic differential equation associated with this model does not fit the standard assumptions in the stochastic optimal control literature, namely sublinear growth, we develop an appropriate version of the dynamic programming principle for the problem under consideration herein, proceeding also to provide a characterization of the optimal harvesting strategies and discuss some qualitative properties of the corresponding value function. [ABSTRACT FROM AUTHOR]

We present an efficient method for solving optimal stopping problems with a probabilistic constraint. The goal is to optimize the expected cumulative cost, but constrained by an upper bound on the probability that the cost exceeds a specified threshold. This probabilistic constraint causes optimal policies to be time-dependent and randomized, however, we show that an optimal policy can always be selected with 'piecewise-monotonic' time-dependence and 'nearly-deterministic' randomization. We prove these properties using the Bellman optimality equations for a Lagrangian relaxation of the original problem. We present an algorithm that exploits these properties for computational efficiency. Its performance and the structure of optimal policies are illustrated on two numerical examples. [ABSTRACT FROM AUTHOR]

We consider the optimal investment and consumption behavior of an agricultural sector that generates income from productive capital and interest payments and that has expenditures due to interest payments for debt. The productivity of capital, the interest rate, and the capital depreciation rate are uncertain. We adapt an existing model from the literature and generalize it to a stochastic depreciation of capital stock. Moreover, we assume a discontinuous evolution of the productivity of capital. The uncertainties are modeled as stochastic processes with discontinuous paths using well-known Wiener processes and simple Poisson processes. Accordingly, the investor's net wealth follows a jump diffusion stochastic differential equation. The goal of maximizing the utility of consumption over an infinite time horizon leads to a stochastic optimal control problem. We choose a hyperbolic absolute risk aversion utility function. To determine the optimal investment and consumption rate, the associated Hamilton-Jacobi-Bellman equation is solved by a separation method, and a verification theorem is applied. [ABSTRACT FROM AUTHOR]

In this paper, we study a stochastic recursive optimal control problem in which the objective functional is described by the solution of a backward stochastic differential equation driven by $$G$$ -Brownian motion. Under standard assumptions, we establish the dynamic programming principle and the related Hamilton-Jacobi-Bellman (HJB) equation in the framework of $$G$$ -expectation. Finally, we show that the value function is the viscosity solution of the obtained HJB equation. [ABSTRACT FROM AUTHOR]

This paper studies the stochastic optimal control problem of finding optimal dividend policies of an insurance company in discrete time with the use of general Lipschitz payoff functions involving indicators of profitability and risk. To construct positional optimal controls and to evaluate the performance indicators, the dynamic programming method is validated. The convergence rate of the successive approximation method in finding generally unbounded Bellman functions is estimated. The Pareto-optimal set of the problem is numerically approximated by so-called barrier-proportional control strategies. [ABSTRACT FROM AUTHOR]

For a sequence of dynamic optimization problems, we aim at discussing a notion of consistency over time. This notion can be informally introduced as follows. At the very first time step t, the decision maker formulates an optimization problem that yields optimal decision rules for all the forthcoming time steps t, t,..., T; at the next time step t, he is able to formulate a new optimization problem starting at time t that yields a new sequence of optimal decision rules. This process can be continued until the final time T is reached. A family of optimization problems formulated in this way is said to be dynamically consistent if the optimal strategies obtained when solving the original problem remain optimal for all subsequent problems. The notion of dynamic consistency, well-known in the field of economics, has been recently introduced in the context of risk measures, notably by Artzner et al. (Ann. Oper. Res. 152(1):5-22, ) and studied in the stochastic programming framework by Shapiro (Oper. Res. Lett. 37(3):143-147, ) and for Markov Decision Processes (MDP) by Ruszczynski (Math. Program. 125(2):235-261, ). We here link this notion with the concept of 'state variable' in MDP, and show that a significant class of dynamic optimization problems are dynamically consistent, provided that an adequate state variable is chosen. [ABSTRACT FROM AUTHOR]

Multiple-Cumulant Cost Density-Shaping (MCCDS) control is proposed for the case when the system is linear and the cost is quadratic. This linear optimal control results from the minimization of an analytic, convex, non-negative function of cost cumulants and target cost cumulants. The MCCDS control allows the designer to shape the initial cost density with respect to a target density approximated by target cost cumulants. A numerical experiment shows that MCCDS control compares favorably with competing control paradigms in terms of official performance measures for inter-story drifts and per-story accelerations used in the first-generation structure benchmark for seismically excited buildings. [ABSTRACT FROM AUTHOR]

This paper is concerned with the stochastic optimal control problem of jump diffusions. The relationship between stochastic maximum principle and dynamic programming principle is discussed. Without involving any derivatives of the value function, relations among the adjoint processes, the generalized Hamiltonian and the value function are investigated by employing the notions of semijets evoked in defining the viscosity solutions. Stochastic verification theorem is also given to verify whether a given admissible control is optimal. [ABSTRACT FROM AUTHOR]