Search

We develop a dual-control method for approximating investment strategies in multidimensional financial markets with convex trading constraints. The method relies on a projection of the optimal solution to an (unconstrained) auxiliary problem to obtain a feasible and near-optimal solution to the original problem. We obtain lower and upper bounds on the optimal value function using convex duality methods. The gap between the bounds indicates the precision of the near-optimal solution. We illustrate the effectiveness of our method in a market with different trading constraints such as borrowing, short-sale constraints and non-traded assets. We also show that our method works well for state-dependent utility functions.

Modern robotic systems are expected to operate robustly in partially unknown environments. This article proposes an algorithm capable of controlling a wide range of high-dimensional robotic systems in such challenging scenarios. Our method is based on the path integral formulation of stochastic optimal control, which we extend with constraint-handling capabilities. Under our control law, the optimal input is inferred from a set of stochastic rollouts of the system dynamics. These rollouts are simulated by a physics engine, placing minimal restrictions on the types of systems and environments that can be modeled. Although sampling-based algorithms are typically not suitable for online control, we demonstrate in this work how importance sampling and constraints can be used to effectively curb the sampling complexity and enable real-time control applications. Furthermore, the path integral framework provides a natural way of incorporating existing control architectures as ancillary controllers for shaping the sampling distribution. Our results reveal that even in cases where the ancillary controller would fail, our stochastic control algorithm provides an additional safety and robustness layer. Moreover, in the absence of an existing ancillary controller, our method can be used to train a parametrized importance sampling policy using data from the stochastic rollouts. The algorithm may thereby bootstrap itself by learning an importance sampling policy offline and then refining it to unseen environments during online control. We validate our results on three robotic systems, including hardware experiments on a quadrupedal robot., The International Journal of Robotics Research, 41 (2), ISSN:0278-3649, ISSN:1741-3176

We study the optimal investment problem in a constrained financial market,where the proportion of borrowed amount to the current wealth level is no more than a given constant. The objective is to maximize the goal-reaching probability before drawdown,namely,the probability that the value of the wealth process reaches the safe level before hitting a lower dynamic barrier. The financial market consists of a risk-free asset and multiple risky assets. By the construction of auxiliary market and convex analysis,we relax the borrowing constraint and investigate the new optimization problem in an auxiliary market,where there is no such borrowing constraint. Then,we find the relationship between the optimal results in auxiliary market and those in constrained market. The explicit expressions for the optimal investment strategy and the maximum goal-reaching probability before drawdown are derived in closed-form. Finally,we provide some numerical examples to show the effect of model parameters on the behaviors of investor.

International audience; We consider a continuous time stochastic optimal control problem under both equality and inequality constraints on the expectation of some functionals of the controlled process. Under a qualification condition, we show that the problem is in duality with an optimization problem involving the Lagrange multiplier associated with the constraints. Then by convex analysis techniques, we provide a general existence result and some a priori estimation of the dual optimizers. We further provide a necessary and sufficient optimality condition for the initial constrained control problem. The same results are also obtained for a discrete time constrained control problem. Moreover, under additional regularity conditions, it is proved that the discrete time control problem converges to the continuous time problem, possibly with a convergence rate. This convergence result can be used to obtain numerical algorithms to approximate the continuous time control problem, which we illustrate by two simple numerical examples.

We consider a stochastic optimal control problem with state variable dynamics described by a stochastic differential equation of diffusive type modulated by a semi-Markov process with a finite state space. The time horizon is both deterministic and finite. Within such setup, we provide a detailed proof of the dynamic programming principle and use it to characterize the value function as a viscosity solution of the corresponding Hamilton-Jacobi-Bellman equation. We illustrate our results with an application to Mathematical Finance: the generalization of Merton's optimal consumption-investment problem to financial markets with semi-Markov switching.

Control based on vision data is a growing field of research and it is widespread in industry. The amount of data in each image and the processing needed to obtain control-relevant information from this data lead to significant delays with a large variability in the control loops. This often causes performance deterioration since in many cases the delay variability is not explicitly addressed in the control design. In this paper, we approach this problem by applying the ideas of recently developed model-based control design methods, which are tailored to address stochastic delays directly, to the motion control of an omnidirectional robot with a vision-based self-localization algorithm. The completion time or delay of the Random Sample Consensus (RANSAC) based localization algorithm is identified as a stochastic random variable with significant variability, illustrating the practical difficulties with data processing. Our main aim is to show that the novel deadline-driven and event-driven control designs significantly outperform a traditional periodic control implementation for a stochastic optimal control performance index.

In this paper, we describe the new OPTCON3 algorithm, which serves to determine approximately optimal policies for stochastic control problems with a quadratic objective function and nonlinear dynamic models. It includes active learning and the dual effect of optimizing policies, whereby optimal policies are used to learn about the stochastics of the dynamic system in addition to their immediate effect on the performance of the system. The OPTCON3 algorithm approximates the nonlinear model with a time-varying linear model and applies a procedure similar to that of Kendrick to the series of linearized models to calculate approximately optimal policies. The results for two simple economic models serve to test the OPTCON3 algorithm and compare it to previous solutions of the stochastic control problem. Initial evaluations show that the OPTCON3 approach may be promising to enhance our understanding of the adaptive economic policy problem under uncertainty.

In this paper, we explore the way in which different bargaining settings affect labour market fluctuations by means of an analytical apparatus that has never been used for this purpose. Specifically, modelling wage negotiations as a problem of stochastic optimal control, we analyze how productivity disturbances shape the dynamics of output, employment, and wages by focusing on the way in which firms’ technology and workers’ preferences interact with the adjustment rules of employment underlying the bargaining process. With a quadratic production function and risk averse workers, we show that wage negotiation outcomes whose employment adjustments go in the direction of the labour demand of the firms match the cyclical behaviour of the involved variables but fail to replicate the observed wage rigidity. By contrast, we show that wage bargaining outcomes whose employment adjustments target the contract curve of two negotiating parties are also able to deliver a strong degree of wage stickiness.

This paper investigates optimal power management of a fuel cell hybrid small unmanned aerial vehicle (sUAV) from the perspective of endurance (time of flight) maximization in a stochastic environment. Stochastic drift counteraction optimal control is exploited to obtain an optimal policy for power management that coordinates the operation of the fuel cell and battery to maximize the expected flight time while accounting for the limits on the rate of change of fuel cell power output and the orientation dependence of fuel cell efficiency. The proposed power management strategy accounts for known statistics in transitions of propeller power and climb angle during the mission, but does not require the exact preview of their time histories. The optimal control policy is generated offline using value iterations implemented in Cython, demonstrating an order of magnitude speedup as compared to MATLAB. It is also shown that the value iterations can be further sped up using a discount factor, but at the cost of decreased performance. Simulation results for a 1.5 kg sUAV are reported that illustrate the optimal coordination between the fuel cell and the battery during aircraft maneuvers, including a turnpike in the battery state of charge (SOC) trajectory. As the fuel cell is not able to support fast changes in power output, the optimal policy is shown to charge the battery to the turnpike value if starting from a low initial SOC value. If starting from a high SOC value, the battery energy is used till a turnpike value of the SOC is reached with further discharge delayed to later in the flight. For the specific scenarios and simulated sUAV parameters considered, the results indicate the capability of up to 2.7 h of flight time.

International audience; In this article, we tackle the problem of a market maker in charge of a book of options on a single liquid underlying asset. By using an approximation of the portfolio in terms of its vega, we show that the seemingly high-dimensional stochastic optimal control problem of an option market maker is in fact tractable. More precisely, when volatility is modeled using a classical stochastic volatility model—e.g. the Heston model—the problem faced by an option market maker is characterized by a low-dimensional functional equation that can be solved numerically using a Euler scheme along with interpolation techniques, even for large portfolios. In order to illustrate our findings, numerical examples are provided.

We provide a rigorous mathematical formulation of Deep Learning (DL) methodologies through an in-depth analysis of the learning procedures characterizing Neural Network (NN) models within the theoretical frameworks of Stochastic Optimal Control (SOC) and Mean-Field Games (MFGs). In particular, we show how the supervised learning approach can be translated in terms of a (stochastic) mean-field optimal control problem by applying the Hamilton&ndash, Jacobi&ndash, Bellman (HJB) approach and the mean-field Pontryagin maximum principle. Our contribution sheds new light on a possible theoretical connection between mean-field problems and DL, melting heterogeneous approaches and reporting the state-of-the-art within such fields to show how the latter different perspectives can be indeed fruitfully unified.

We correct assumption (4.6) in Theorem 4.1 in [F. Gozzi and F. Masiero, SIAM J. Control Optim., 55 (2017), pp. 2981--3012], and we consequently change the proof of Theorem 4.1. We correct the proof...

We present the design of a motion planning algorithm that ensures safety for an autonomous vehicle. In particular, we consider a multimodal distribution over uncertainties; for example, the uncertain predictions of future trajectories of surrounding vehicles reflect discrete decisions, such as turning or going straight at intersections. We develop a computationally efficient, scenario-based approach that solves the motion planning problem with high confidence given a quantifiable number of samples from the multimodal distribution. Our approach is based on two preprocessing steps, which 1) separate the samples into distinct clusters and 2) compute a bounding polytope for each cluster. Then, we rewrite the motion planning problem approximately as a mixed-integer problem using the polytopes. We demonstrate via simulation on the nuScenes dataset that our approach ensures safety with high probability in the presence of multimodal uncertainties, and is computationally more efficient and less conservative than a conventional scenario approach., Comment: Published in IEEE Control Systems Letters

International audience; In most over-the-counter (OTC) markets, a small number of market makers provide liquidity to other market participants. More precisely, for a list of assets, they set prices at which they agree to buy and sell. Market makers face therefore an interesting optimization problem: they need to choose bid and ask prices for making money while mitigating the risk associated with holding inventory in a volatile market. Many market-making models have been proposed in the academic literature, most of them dealing with single-asset market making whereas market makers are usually in charge of a long list of assets. The rare models tackling multiasset market making suffer however from the curse of dimensionality when it comes to the numerical approximation of the optimal quotes. The goal of this paper is to propose a dimensionality reduction technique to address multiasset market making by using a factor model. Moreover, we generalize existing market-making models by the addition of an important feature: the existence of different transaction sizes and the possibility for the market makers in OTC markets to answer different prices to requests with different sizes.

This note corrects Lemma 3.7 in our paper (Ann. Probab. 44 (2016) 3740–3803). The main results of the paper remain correct as stated.

International audience; When firms want to buy back their own shares, they have a choice between several alternatives. If they often carry out open market repurchase, they also increasingly rely on banks through complex buyback contracts involving option components, e.g. accelerated share repurchase contracts, VWAP-minus profit-sharing contracts, etc. The entanglement between the execution problem and the option hedging problem makes the management of these contracts a difficult task that should not boil down to simple Greek-based risk hedging, contrary to what happens with classical books of options. In this paper, we propose a machine learning method to optimally manage several types of buyback contract. In particular, we recover strategies similar to those obtained in the literature with partial differential equation and recombinant tree methods and show that our new method, which does not suffer from the curse of dimensionality, enables to address types of contract that could not be addressed with grid or tree methods.

This letter presents a stochastic model predictive control approach (MPC) for linear discrete-time systems subject to unbounded and correlated additive disturbance sequences, which makes use of the scenario approach for offline computation of probabilistic reachable sets. These sets are used in a tube-based MPC formulation, resulting in low computational requirements. Using a recently proposed MPC initialization scheme and nonlinear tube controllers, we provide recursive feasibility and closed-loop chance constraint satisfaction, as well as hard input constraint guarantees, which are typically challenging in tube-based formulations with unbounded noise. The approach is demonstrated in simulation for the control of an overhead crane system., IEEE Control Systems Letters, 4 (2), ISSN:2475-1456

National audience; A classical problem in stochastic ergodic control consists of studying the limit behavior of the optimal value of a discounted integral in infinite horizon (the so called Abel mean of an integral cost) as the discount factor $\lambda$ tends to zero or the value defined with a Cesàro mean of an integral cost when the horizon $T$ tends to $+ \infty$. We investigate the possible limits in the norm of uniform convergence topology of values defined through Abel means or Ceàro means when $ \lambda \to 0^+ $ and $T \to + \infty $, respectively. Here we give two types of new representation formulas for the accumulation points of the values when the averaging parameter converges. We show that there is only one possible accumulation point which is the same for Abel means or Cesàro means. The first type of representation formula is based on probability measures on the product of the state space and the control state space, which are limits of occupational measures. The second type of representation formula is based on measures which are the projection of invariant measure on the space of relaxed controls. We also give a result comparing the both sets of measures involved in both classes of representation formulas. An important consequence of the representation formulas is the existence of the limit value when one has the equicontinuity property of Abel or Cesàro mean values. This is the case, for example, for nonexpansive stochastic control systems. In the end some insightful examples are given which help to better understand the results.

An event-triggered control (ETC) strategy is consistent if it achieves a better closed-loop performance than that of traditional periodic control for the same average transmission rate and does not generate transmissions in the absence of disturbances. In this letter, we propose a consistent ETC strategy for discrete-time linear systems with partial state information and Gaussian noise and disturbances when the performance is measured by an average quadratic cost, just as in the linear quadratic Gaussian (LQG) framework. This strategy incorporates a scheduler determining transmissions based on the error between two state estimates, which are provided by a stationary Kalman filter at the sensors/scheduler side and an estimator at the controller/actuators side relying on previously transmitted data. Through a numerical example, we show that the proposed strategy can achieve impressive performance gains with respect to periodic control for the same average transmission rate.

We present the Forest of Stochastic Trees (FOST) method for pricing multiple exercise options by simulation. The proposed method uses stochastic trees in place of binomial trees in the Forest of Trees algorithm originally proposed to value swing options, hence extending that method to allow for a multi-dimensional underlying process. The FOST can also be viewed as extending the stochastic tree method for valuing (single exercise) American-style options to multiple exercise options. The proposed valuation method results in positively- and negatively-biased estimators for the true option value. We prove the sign of the estimator bias and show that these estimators are consistent for the true option value. This method is of particular use in cases where there is potentially a large number of assets underlying the contract and/or the underlying price process depends on multiple risk factors. Numerical results are presented to illustrate the method.

Optimal control is a prominent approach in robotics and movement neuroscience, among other fields of science. Methods for deriving optimal choices of action have been classically devised either in deterministic or stochastic settings. Here, we consider a setting in-between that retains the stochastic aspect of the controlled system but assumes deterministic open-loop control actions. The rationale stems from observations about the neural control of movement which highlighted that relatively stable behaviors can be achieved without feedback circuitry, via open-loop motor commands adequately tuning the mechanical impedance of the neuromusculoskeletal system. Yet, effective methods for deriving optimal open-loop controls for stochastic systems are lacking overall. This work presents a continuous-time approach based on statistical linearization techniques for the efficient computation of optimal open-loop controls for a broad class of stochastic optimal control problems. We first show that non-trivial departure from the optimal solutions of classical deterministic and stochastic approaches may arise for simple synthetic examples, thereby stressing the originality of the framework. We then exemplify its potential relevance to the planning of biological movement by showing that a well-known phenomenon in motor control, referred to as muscle co-contraction, occurs naturally. More generally, this stochastic optimal control framework may be suited to other fields where the design of optimal open-loop actions is relevant.

With the penetration of renewable generation, electric vehicles and other random factors in power systems, the stochastic disturbances are increasing significantly, which are necessary to be handled for guarantying the security of systems. A novel stochastic optimal control strategy is proposed in this paper to reduce the impact of such stochastic continuous disturbances on power systems. The proposed method is effective in solving the problems caused by the stochastic continuous disturbances and has two significant advantages. First, a simplified and effective solution is proposed to analyze the system influenced by the stochastic disturbances. Second, a novel optimal control strategy is proposed in this paper to effectively reduce the impact of stochastic continuous disturbances. To be specific, a novel excitation controlled power systems model with stochastic disturbances is built in the quasi-generalized Hamiltonian form, which is further simplified into a lower-dimension model through the stochastic averaging method. Based on this It&ocirc, equation, a novel optimal control strategy to achieve the intra-region probability maximization is established for power systems by using the dynamic programming method. Finally, the intra-region probability increases in controlled systems, which confirms the effectiveness of the proposed control strategy. The proposed control method has advantages on controlling the fluctuation of system state variables within a desired region under the influence of stochastic disturbances, which means improving the security of stochastic systems. With more stochasticity in the future, the proposed control method based on the stochastic theory will play a novel way to relieve the impact of stochastic disturbances.

While numerical approaches to solve financial and actuarial stochastic optimization problems are usually based on dynamic programming, we explore an approach through a stochastic maximum principle formulation followed by the use of least squares regression to determine the optimal control policy. We show that this methodology can be applied to a number of realistic financial and actuarial problems of increasing complexity to highlight potential strengths and applications of this approach. We cast a direct connection between this approach and the stochastic duality approach to stochastic optimization. In particular, we discuss the potential improvements which can derive from this reformulation in terms of numerical precision and in order to provide bounds to control the simulation errors. The critical numerical issue is shown to be the numerical computation of conditional expectations which is performed applying the approach of Longstaff and Schwartz [1].

In the present paper we derive the existence and uniqueness of a solution for the optimal control problem determined by a stochastic FitzHugh-Nagumo equation with recovery variable. In particular due the cubic non-linearity in the drift coefficients, standard techniques cannot be applied so that the Ekeland's variational principle has to be exploited.

Several studies in the literature have shown the potential energy savings emerging from the cooperative management of the aggregated building energy demands. Sophisticated predictive control schemes have recently been developed that achieve these gains by exploiting the energy generation, conversion and storage equipment shared by the building community. A common difficulty with all these methods is integrating knowledge about the long term evolution of the disturbances affecting the system dynamics (e.g. ambient temperature and solar radiation). In this context, the seasonal storage capabilities of the systemare difficult to be optimally managed. This paper addresses this issue by exploiting available historical data to (i) construct bounds that confine with high probability the optimal charging trajectory of the seasonal storage, and (ii) generate a piece-wiseaffine approximation of the value function of the energy stored in the seasonal storage at each time step. Using these bounds and value functions, we formulate a multistage stochastic optimization problem to minimize the total energy consumption of the system. In a numerical study based on a realistic system configuration, the proposed method is shown to operate the system close to global optimality., IEEE Control Systems Letters, 1 (2), ISSN:2475-1456

Mathematical formulation of a stochastic hepatitis B virus (HBV) model with the application of optimal control and randomly noise transmission has been focused in this paper. For the ease of understanding, the model is divided into four different classes of healthy or susceptible, acute infected, chronically infected and the class of the recovered population. All four cases have been perturbed by the white noise. By applying optimal control techniques, we investigated both deterministic and stochastic model for control. The approximate solution method of the deterministic model has been used in the numerical solution of the required stochastic model. The simulation of both models has been drawn against the given data and compared with each other.

This study analyzes the impact of contagion between financial and non-life insurance markets on the asset–liability management policy of an insurance company. The indirect dependence between these markets is modeled by assuming that the assets return and non-life insurance claims are led respectively by time-changed Brownian and jump processes, for which stochastic clocks are integrals of mutually self-exciting processes. This model exhibits delayed co-movements between financial and non-life insurance markets, caused by events like natural disasters, epidemics, or economic recessions.

We present a novel technique to solve the problem of managing optimally a pumped hydroelectric storage system. This technique relies on representing the system as a stochastic optimal control problem with state constraints, these latter corresponding to the finite volume of the reservoirs. Following the recent level-set approach presented in O. Bokanowski, A. Picarelli, H. Zidani, State-constrained stochastic optimal control problems via reachability approach, SIAM J. Control and Optim. 54 (5) (2016), we transform the original constrained problem in an auxiliary unconstrained one in augmented state and control spaces, obtained by introducing an exact penalization of the original state constraints. The latter problem is fully treatable by classical dynamic programming arguments.

International audience; In corporate bond markets, which are mainly OTC markets, market makers play a central role by providing bid and ask prices for bonds to asset managers. Determining the optimal bid and ask quotes that a market maker should set for a given universe of bonds is a complex task. The existing models, mostly inspired by the Avellaneda-Stoikov model, describe the complex optimization problem faced by market makers: proposing bid and ask prices for making money out of the difference between them while mitigating the market risk associated with holding inventory. While most of the models only tackle one-asset market making, they can often be generalized to a multi-asset framework. However, the problem of solving the equations characterizing the optimal bid and ask quotes numerically is seldom tackled in the literature, especially in high dimension. In this paper, we propose a numerical method for approximating the optimal bid and ask quotes over a large universe of bonds in a model à la Avellaneda–Stoikov. As classical finite difference methods cannot be used in high dimension, we present a discrete-time method inspired by reinforcement learning techniques, namely, a model-based deep actor-critic algorithm.

In a random environment, the growth of a population subjected to harvesting can be described by a stochastic differential equation.We consider a population with natural growth following a Gompertz model with harvesting proportional to population size and to the exerted fishing effort. The main goal of this work is to compare the performance of three fishing policies: one with variable effort, named optimal policy, the second based on variable effort but with small periods with constant effort, named stepwise policy, and another with constant effort, denoted by optimal sustainable policy.We will show that the first policy is inapplicable from the practical point of view, whereas the second one, although being suboptimal, is applicable but has some shortcomings common to the optimal policy. On the contrary, the constant effort policy is easily implemented and predicts the sustainability of the population as well as the existence of a stationary density for its size. The performance of the policies will be assessed by the profit obtained over a finite time horizon. We also show how changes in the parameter values affect the profit differences between the variable effort policy and the constant effort policy.

Mass vaccination campaigns play major roles in the war against epidemics. Such prevention strategies cannot always reach their goals significantly without the help of media and awareness campaigns used to prevent contacts between susceptible and infected people. Feelings of fear, infodemics, and misconception could lead to some fluctuations of such policies. In addition to the vaccination strategy, the movement restriction approach is essential because of the factor of mobility or travel. However, anti-epidemic border measures may also be disturbed if some infected travelers manage to escape and infiltrate into a safer region. In this paper, we aim to study infection dynamics related to the spatial spread of an epidemic in interconnected regions in the presence of random perturbations caused by the three above-mentioned reasons. Therefore, we devise a stochastic multi-region epidemic model in which contacts between susceptible and infected populations, vaccination-based and movement restriction optimal control approaches are all assumed to be unpredictable, and then, we discuss the effectiveness of such policies. In order to reach our goal, we employ a stochastic maximum principle version for noised systems, state and prove the sufficient and necessary conditions of optimality, and finally provide the numerical results obtained using a stochastic progressive-regressive schemes method.

A simplified stochastic control model for optimization of logistic dynamics with the control-dependent carrying capacity, which is motivated by a recent algae population management problem in the river environment, is presented. Solving the optimization problem reduces to finding a solution to a non-local first-order differential equation called tt-Jacobi-Bellman (HJB) equation. It is shown that the HJB equation has a unique viscosity solution and that the solution can be approximated with a finite difference scheme. Asymptotic estimates of the solution and the optimal control are derived and compared with numerical solutions. Finally, parameter dependence of the optimal control is examined numerically with implications to river environmental management.

Based on a generative model (GM) and beliefs over hidden states, the free energy principle (FEP) enables an agent to sense and act by minimizing a free energy bound on Bayesian surprise, i.e., the negative logarithm of the marginal likelihood. Inclusion of desired states in the form of prior beliefs in the GM leads to active inference (ActInf). In this work, we aim to reveal connections between ActInf and stochastic optimal control. We reveal that, in contrast to standard cost and constraint-based solutions, ActInf gives rise to a minimization problem that includes both an information-theoretic surprise term and a model-predictive control cost term. We further show under which conditions both methodologies yield the same solution for estimation and control. For a case with linear Gaussian dynamics and a quadratic cost, we illustrate the performance of ActInf under varying system parameters and compare to classical solutions for estimation and control.

[Departement_IRSTEA]Territoires [ADD1_IRSTEA]Bioéconomie territoriale; International audience; By means of stochastic optimal control, this paper aims at studying the shadow pricing of renewable natural resources in uncertainty. Two cases are considered, respectively centralized and decentralized control processes. The decentralized control is in the form of a stochastic control of the state vector distributed among several agents. In both cases, the optimal control minimizing the cost function, which is a decreasing function of time, corresponds to the real option valuation. The latter is a cost-effective optional investment in the resource stock preservation in uncertainty. The results obtained from numerical simulations show coherence with those encountered in the literature on option pricing.

This article considers the problem of optimal liquidation of a position in a risky security quoted in a financial market, where price evolution are risky and trades have an impact on price as well ...

© 1963-2012 IEEE. This paper is concerned with learning and stochastic control in physical systems that contain unknown input signals. These unknown signals are modeled as Gaussian processes (GP) with certain parameterized covariance structures. The resulting latent force models can be seen as hybrid models that contain a first-principle physical model part and a nonparametric GP model part. We briefly review the statistical inference and learning methods for this kind of models, introduce stochastic control methodology for these models, and provide new theoretical observability and controllability results for them.

The COVID-19 pandemic illustrates the importance of treatment-related decision making in populations. This article considers the case where the transmission rate of the disease as well as the efficiency of treatments is subject to uncertainty. We consider two different regimes, or submodels, of the stochastic SIR model, where the population consists of three groups: susceptible, infected and recovered and dead. In the first regime the proportion of infected is very low, and the proportion of susceptible is very close to 100the proportion of infected is moderate, but not negligible. We show that the first regime corresponds almost exactly to a well-known problem in finance, the problem of portfolio and consumption decisions under mean-reverting returns (Wachter, JFQA 2002), for which the optimal control has an analytical solution. We develop a perturbative solution for the second problem. To our knowledge, this paper represents one of the first attempts to develop analytical/perturbative solutions, as opposed to numerical solutions to stochastic SIR models.

International audience; We consider the management of a single hydroelectric dam, subject to uncertain inflows and electricity prices and to a so-called “tourism constraint”: the water storage level must be high enough during the tourist season with high enough probability. We cast the problem in the stochastic optimal control framework: we search at each time t the optimal control as a function of the available information at t. We lay out two approaches. First, we formulate a chance-constrained stochastic optimal control problem: we maximize the expected gain while guaranteeing a minimum storage level with a minimal prescribed probability level. Dualizing the chance constraint by a multiplier, we propose an iterative algorithm alternating additive dynamic programming and update of the multiplier value “à la Uzawa”. Our numerical results reveal that the random gain is very dispersed around its expected value; in particular, low gain values have a relatively high probability to materialize. This is why, to put emphasis on these low values, we outline a second approach. We propose a so-called stochastic viability approach that focuses on jointly guaranteeing a minimum gain and a minimum storage level during the tourist season. We solve the corresponding problem by multiplicative dynamic programming. To conclude, we discuss and compare the two approaches.Keywords

This paper is concerned with the existence and uniqueness of solution for the optimal control problem governed by the stochastic FitzHugh–Nagumo equation driven by a Gaussian noise. First-order conditions of optimality are also obtained.

Automated control of blood glucose (BG) concentration with a fully automated artificial pancreas will certainly improve the quality of life for insulin-dependent patients. Closed-loop insulin delivery is challenging due to inter- and intra-patient variability, errors in glucose sensors and delays in insulin absorption. Responding to the varying activity levels seen in outpatients, with unpredictable and unreported food intake, and providing the necessary personalized control for individuals is a challenging task for existing control algorithms. A novel approach for controlling glycemic variability using simulation-based learning is presented. A policy iteration algorithm that combines reinforcement learning with Gaussian process approximation is proposed. To account for multiple sources of uncertainty, a control policy is learned off-line using an Ito´s stochastic model of the glucose-insulin dynamics. For safety and performance, only relevant data are sampled through Bayesian active learning. Results obtained demonstrate that a generic policy is both safe and efficient for controlling subject-specific variability due to a patient´s lifestyle and its distinctive metabolic response. Fil: de Paula, Mariano. Universidad Nacional del Centro de la Provincia de Buenos Aires. Facultad de Ingeniería Olavarria. Departamento de Electromecánica. Grupo INTELYMEC; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Tandil. Centro de Investigaciones en Física e Ingeniería del Centro de la Provincia de Buenos Aires; Argentina Fil: Avila, Luis Omar. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Santa Fe. Instituto de Desarrollo y Diseño (i); Argentina Fil: Martinez, Ernesto Carlos. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Santa Fe. Instituto de Desarrollo y Diseño (i); Argentina

A conventional approach to solving stochastic optimal control problems with time-dependent uncertainties involves the use of the stochastic maximum principle (SMP) technique. For large-scale problems, however, such an algorithm frequently leads to convergence complexities when solving the two-point boundary value problem resulting from the optimality conditions. An alternative approach consists of using continuous random variables to capture uncertainty through sampling-based methods embedded within an optimization strategy for the decision variables, such a technique may also fail due to the computational intensity involved in excessive model calculations for evaluating the objective function and its derivatives for each sample. This paper presents a new approach to solving stochastic optimal control problems with time-dependent uncertainties based on BONUS (Better Optimization algorithm for Nonlinear Uncertain Systems). The BONUS has been used successfully for non-linear programming problems with static uncertainties, but we show here that its scope can be extended to the case of optimal control problems with time-dependent uncertainties. A batch reactor for biodiesel production was used as a case study to illustrate the proposed approach. Results for a maximum profit problem indicate that the optimal objective function and the optimal profiles were better than those obtained by the maximum principle.

We study the gap between the state pension provided by the Italian pension system pre-Dini reform and post-Dini reform. The goal is to fill the gap between the old and the new pension by joining a defined contribution pension scheme and adopting an optimal investment strategy that is target-based. We find that it is possible to cover, at least partially, this gap with the additional income of the pension scheme, especially in the presence of late retirement and in the presence of stagnant careers. Workers with dynamic careers and workers who retire early are those who are most penalised by the reform. Results are intuitive and in line with previous studies on the subject.

In this paper we investigate the optimal investment strategy for a defined contribution (DC) pension plan during the decumulation phrase which is risk-averse and pays close attention to inflation risk. The plan aims to maximize the expected constant relative risk aversion (CRRA) utility from the terminal wealth by investing the wealth in a financial market consisting of an inflation-indexed bond, an ordinary zero coupon bond and a risk-free asset. We derive the optimal investment strategy in closed-form using the dynamic programming approach by solving the corresponding Hamilton-Jacobi-Bellman (HJB) equation. Our theoretical and numerical results reveal that under some rational assumptions, an inflation-indexed bond do has significant advantage to hedge inflation risk.

The goal of this paper is to study potential applicability and performance of approximate dynamic programming (ADP) for building control problems. It is well known that occupants' stochastic behavior affects the thermal dynamics of building spaces. Incorporating occupant interactions in building control system designs is the main focus of this work. We apply ADP to stochastic optimal control designs for illustrative scenarios of occupant-building interactions and demonstrate its validity through a simulation study.

In the present paper we derive, via a backward induction technique, and ad hoc maximum principle for an optimal control problem with multiple random terminal times. Therefore we apply the aforementioned result to the case of a linear quadratic controller, providing solutions for the optimal control in terms of Riccati backward SDE with random terminal time. Eventually all the above results are applied to a system of interconnected banks.

An optimal production flow control problem of a make-to-stock manufacturing firm with price, cost, and demand uncertainty is studied. The objective of the flow rate control problem is maximizing the average profit that is the difference between the expected revenue and the expected production, inventory holding, and backlog costs. The uncertainties in the system are captured jointly in discrete environment states. In each environment state, the price, cost, and demand take different levels. The transitions between different environment states evolve according to a time-homogenous Markov chain. By using a continuous flow model, the optimal production policy is stated as a state-dependent hedging policy. The performance of the system where the production cost alternates between a high and a low cost level and the demand is either constant or also alternates between a high and a low level is analyzed under the double-hedging policy. According to this policy, the producer produces only when the cost is low and the surplus is between the two hedging levels. However when the backlog is below the lower hedging level, the producer produces with the maximum capacity regardless of the cost. The effects of production cost, production capacity, demand variability, and the dependence of the demand and the cost on the performance of the system are analyzed analytically and numerically. It is shown that controlling the production rate optimally allows producers respond to the fluctuations in price, cost, and demand in an effective way and maximize their profits., NA

Dealing with meteorological uncertainty poses a major challenge in air traffic management (ATM). Convective weather (commonly referred to as storms or thunderstorms) in particular represents a significant safety hazard that is responsible for one quarter of weather-related ATM delays in the US. With commercial air traffic on the rise and the risk of potentially critical capacity bottlenecks looming, it is vital that future trajectory planning tools are able to account for meteorological uncertainty. We propose an approach to model the uncertainty inherent to forecasts of convective weather regions using statistical analysis of state-of-the-art forecast data. The developed stochastic storm model is tailored for use in an optimal control algorithm that maximizes the probability of reaching a waypoint while avoiding hazardous storm regions. Both the aircraft and the thunderstorms are modeled stochastically. The performance of the approach is illustrated and validated through simulated case studies based on recent nowcast data and storm observations., Comment: 13 pages, journal

In this paper, we present connections between recent developments on the linearly-solvable stochastic optimal control framework with early work in control theory based on the fundamental dualities between free energy and relative entropy. We extend these connections to nonlinear stochastic systems with non-affine controls by using the generalized version of the Feynman–Kac lemma. We present alternative formulations of the linearly-solvable stochastic optimal control framework and discuss information theoretic and thermodynamic interpretations. On the algorithmic side, we present iterative stochastic optimal control algorithms and applications to nonlinear stochastic systems. We conclude with an overview of the frameworks presented and discuss limitations, differences and future directions.

Energy saving in production plants is becoming more and more relevant due to the pressure from governments to contain the environmental impact of manufacturing and from companies to reduce costs. One of the measures for saving energy is the implementation of control strategies that reduce energy consumption during the machine idle periods. This paper proposes a framework that integrates different control policies for switching the machine off when production is not critical and on when the part flow has to be resumed. A general policy is formalized by modelling explicitly the energy consumed at each machine state. The policy parameters that minimize the requested machine expected energy are provided analytically for general distributions. Numerical results are based on data acquired with dedicated experimental measurements on a real machining centre, and a comparison with the most common practices in manufacturing is also reported.

In this paper we check the credit rating of firms applied for a loan. In this regard we introduce a model, named Dynamic Slack-Based Measure (DSBM) for measuring credit rating of applicant companies. Selection of financial ratios that represent the financial state of a company -in the best possible way- is one of the most challenging parts of any credit rating analysis. At first, ranking needs to identify the appropriate variables. Therefore we introduce five financial variables to provide a ranking. As noted above, we assess the performance of these firms. Then we introduce the dynamic model of SBM and theorems, also we discuss the overall structure of DSBM. Then we will present the implementation and the simulation model. After that, we propose a stochastic controlled dynamic system model to express the optimal strategy. Banks expect companies selected with DSBM model, act in accordance with this strategy. This stochastic dynamic system is originated from the balance sheets of firms applying for a loan. Finally we evaluate the performance of the system and strategy problem.