Search

We study an optimal control problem for a stochastic model of tumour growth with drug application. This model consists of three stochastic hyperbolic equations describing the evolution of tumour cells. It also includes two stochastic parabolic equations describing the diffusions of nutrient and drug concentrations. Since all systems are subject to many uncertainties, we have added stochastic terms to the deterministic model to consider the random perturbations. Then, we have added control variables to the model according to the medical concepts to control the concentrations of drug and nutrient. In the optimal control problem, we have defined the stochastic and deterministic cost functions and we have proved the problems have unique optimal controls. For deriving the necessary conditions for optimal control variables, the stochastic adjoint equations are derived. We have proved the stochastic model of tumour growth and the stochastic adjoint equations have unique solutions. For proving the theoretical results, we have used a change of variable which changes the stochastic model and adjoint equations (a.s.) to deterministic equations. Then we have employed the techniques used for deterministic ones to prove the existence and uniqueness of optimal control. [ABSTRACT FROM AUTHOR]

Problem definition: More than 99% of the new power generation capacity to be installed in the United States from 2023 to 2050 will be powered by wind, solar, and natural gas. Additionally, large-scale battery systems are planned to support power systems. It is paramount for policymakers and electric utilities to deepen the understanding of the operational and investment relations among renewable, flexible (natural gas-powered), and storage capacities. In this paper, we optimize both the joint operations and investment mix of these three types of resources, examining whether they act as investment substitutes or complements. Methodology/results: Using stochastic control theory, we identify and prove the structure of the optimal storage control policy, from which we determine various pairs of charging and discharging operations. We find that whether storage complements or substitutes other resources hinges on the operational pairs involved and whether executing these pairs is constrained by charging or discharging. Through extensive numerical analysis using data from a Florida utility, government agencies, and industry reports, we demonstrate how storage operations drive the investment relations among renewable, flexible, and storage capacities. Managerial implications: Storage and renewables substitute each other in meeting peak demand; storage complements renewables by storing surplus renewable output; renewables complement storage by compressing peak periods, facilitating peak shaving and displacement of flexible capacity. These substitution and complementary effects often coexist, and the dominant effect can alternate as costs change. A thorough understanding of these relations at both operational and investment levels empowers decision makers to optimize energy infrastructure investments and operations, thereby unlocking their full potential. Funding: This research was supported in part by Lilly Endowment, Inc., through its support for the Indiana University Pervasive Technology Institute. This research was also supported by Kelley School of Business, Indiana University, and Haslam College of Business, University of Tennessee. O. Q. Wu thanks Grant Thornton for their generous support. Supplemental Material: The online appendix is available at https://doi.org/10.1287/msom.2023.0068. [ABSTRACT FROM AUTHOR]

This paper is concerned with the turnpike property for a class of stochastic linear-quadratic (LQ, for short) optimal control problems with periodic coefficients. The stability and stabilizability of the control system are studied, followed by the discussion of the existence and uniqueness of periodic solutions. A deterministic periodic LQ problem is introduced and solved, whose optimal pair, together with a pair of correction processes, serves as the turnpike limit of the stochastic problem. It is shown that the turnpike limit is periodic in the distribution sense. In the special case of constant coefficients, the turnpike limit turns out to have stationary distributions, with the expectation being the solution to a static optimization problem. [ABSTRACT FROM AUTHOR]

This paper presents convergence analysis of a novel data-driven feedback control algorithm designed for generating online controls based on partial noisy observational data. The algorithm comprises a particle filter-enabled state estimation component, estimating the controlled system's state via indirect observations, alongside an efficient stochastic maximum principle-type optimal control solver. By integrating weak convergence techniques for the particle filter with convergence analysis for the stochastic maximum principle control solver, we derive a weak convergence result for the optimization procedure in search of optimal data-driven feedback control. Numerical experiments are performed to validate the theoretical findings. [ABSTRACT FROM AUTHOR]

This paper deals with finite‐horizon minimum‐variance and covariance steering problems subject to constraints. The goal of the minimum variance problem is to steer the state mean of an uncertain system to a prescribed vector while minimizing the trace of its terminal state covariance whereas the goal in the covariance steering problem is to steer the covariance matrix of the terminal state to a prescribed positive definite matrix. The paper proposes a solution approach that relies on a stochastic version of the affine disturbance feedback control parametrization. In this control policy parametrization, the control input at each stage is expressed as an affine function of the history of disturbances that have acted upon the system. It is shown that this particular parametrization reduces the stochastic optimal control problems considered in this paper into tractable convex programs or difference of convex functions programs with essentially the same decision variables. In addition, the paper proposes a variation of this control parametrization that relies on truncated histories of past disturbances, which allows for sub‐optimal controllers to be designed that strike a balance between performance and computational cost. The suboptimality of the truncated policies is formally analyzed and closed form expressions are provided for the performance loss due to the use of the truncation scheme. Finally, the paper concludes with a comparative analysis of the truncated versions of the proposed policy parametrization and other standard policy parametrizations through numerical simulations. [ABSTRACT FROM AUTHOR]

We consider a singularly perturbed system of stochastic differential equations proposed by Chaudhari et al. (Res. Math. Sci. 2018) to approximate the entropic gradient descent in the optimization of deep neural networks via homogenization. We embed it in a much larger class of two-scale stochastic control problems and rely on convergence results for Hamilton--Jacobi--Bellman equations with unbounded data proved recently by ourselves (ESAIM Control Optim. Calc. Var. 2023). We show that the limit of the value functions is itself the value function of an effective control problem with extended controls and that the trajectories of the perturbed system converge in a suitable sense to the trajectories of the limiting effective control system. These rigorous results improve the understanding of the convergence of the algorithms used by Chaudhari et al., as well as of their possible extensions where some tuning parameters are modeled as dynamic controls. [ABSTRACT FROM AUTHOR]

The fashion apparel industry is facing an increasingly growing demand, compounded by the short sales lifecycle and strong seasonality of clothing, posing significant challenges to inventory management in the retail sector. Despite some retailers like Uniqlo and Zara implementing inventory management and dynamic pricing strategies, challenges persist due to the dynamic nature of fashion trends and the stochastic factors affecting inventory. To address these issues, we construct a mathematical model based on the mathematical expression of the deterministic fashion level function, where the geometric Brownian motion, widely applied in finance, is initially utilized in the stochastic fashion level function. Drawing on research findings from deteriorating inventory management and stochastic optimization, we investigate the fluctuation of inventory levels, optimal dynamic pricing, optimal production rates, and profits—four crucial indicators—via Pontryagin's maximum principle. Analytical solutions are derived, and the numerical simulation is provided to verify and compare the proposed model with deterministic fashion level function models. The model emphasizes the importance of considering stochastic factors in decision-making processes and provides insights to enhance profitability, inventory management, and sustainable consumption in the fashion product industry. [ABSTRACT FROM AUTHOR]

Information drives trading and hence affects price dynamics. We study how an informed trader optimally trades, how multiple traders compete with each other, and how their competition affects price dynamics as well as their inventories in the linear price impact model framework. We formulate the informed trading problem as a stochastic optimal control problem and a stochastic differential game in different cases. By virtue of different methods, problems are solved explicitly in all cases. In the case of a single trader, the risk-averse informed trader trades less in the beginning and then speeds up to reduce exposure to the volatility of dynamic information. In addition, the risk-averse trader reveals information more slowly and thus contributes less to price efficiency. When multiple risk-neutral traders compete with information, regardless of the competition structure, they tend to trade more rapidly in the early stage to fill orders at a good price. Competition and the presence of a leader help promote price efficiency, and competition also leads to the empirically observed mean-reverting behavior of price dynamics and order flows. Furthermore, among risk-averse informed traders, inventory management triggers predatory trading and active information leakage during informed trading, and the positions of traders with the same level of risk aversion tend to be identical and mean-reverting to zero under multiple rounds of competition. [ABSTRACT FROM AUTHOR]

We present a two-country model (North and South) that describes the phenomenon of offshoring and reshoring. The model is a continuous time-controlled Markov chain with binary states. The main trade-off involves production costs and transaction costs between one country and another. In the first part of this paper, we identify the key parameters of the model: the difference in unit production costs between the two countries considered, the marginal cost of transitioning between countries, and the incentive paid by the North country to all companies that have not relocated at the end of the planning interval. The final goal of our paper is to understand how national tax incentives can influence this process. [ABSTRACT FROM AUTHOR]

We investigate a stochastic differential game in which a major player has private information (the knowledge of a random variable), which she discloses through her control to a population of small players playing in a Nash mean field game equilibrium. The major player's cost depends on the distribution of the population, while the cost of the population depends on the random variable known by the major player. We show that the game has a relaxed solution and that the optimal control of the major player is approximatively optimal in games with a large but finite number of small players. [ABSTRACT FROM AUTHOR]

In this manuscript we consider a class of optimal control problems of stochastic differential delay equations. First, we rewrite the problem in a suitable infinite-dimensional Hilbert space. Then, using the dynamic programming approach, we characterize the value function of the problem as the unique viscosity solution of the associated infinite-dimensional Hamilton-Jacobi-Bellman equation. Finally, we prove a C1,α -partial regularity of the value function. We apply these results to path dependent financial and economic problems (Merton-like portfolio problem and optimal advertising). [ABSTRACT FROM AUTHOR]

In this paper we explore optimal liquidation in a market populated by a number of heterogeneous market makers that have limited inventory-carrying and risk-bearing capacity. We derive a reduced form model for the dynamics of their aggregated inventory considering a proper scaling limit. The resulting price impact profile is shown to depend on the characteristics and relative importance of their inventories. The model is flexible enough to reproduce the empirically documented power law behavior of the price impact function. For any choice of the market makers characteristics, optimal execution within this modeling approach can be recast as a linear-quadratic stochastic control problem. The value function and the associated optimal trading rate can be obtained semi-explicitly subject to solving a differential matrix Riccati equation. Numerical simulations are conducted to illustrate the performance of the resulting optimal liquidation strategy in relation to standard benchmarks. Remarkably, they show that the increase in performance is determined by a substantial reduction of higher order moment risk. [ABSTRACT FROM AUTHOR]

We consider optimal control problems for a system governed by a stochastic differential equation driven by a d-dimensional Brownian motion where both the drift and the diffusion coefficient are controlled. It is well known that without additional convexity conditions the strict control problem does not admit an optimal control. To overcome this difficulty, we consider the relaxed model, in which admissible controls are measure-valued processes and the relaxed state process is governed by a stochastic differential equation driven by a continuous orthogonal martingale measure. This relaxed model admits an optimal control that can be approximated by a sequence of strict controls by the so-called chattering lemma. We establish optimality necessary conditions, in terms of two adjoint processes, extending Peng's maximum principle to relaxed control problems. We show that relaxing the drift and diffusion martingale parts directly as in deterministic control does not lead to a true relaxed model as the obtained controlled dynamics is not continuous in the control variable. [ABSTRACT FROM AUTHOR]

Sampling rare events in metastable dynamical systems is often a computationally expensive task and one needs to resort to enhanced sampling methods such as importance sampling. Since we can formulate the problem of finding optimal importance sampling controls as a stochastic optimization problem, this then brings additional numerical challenges and the convergence of corresponding algorithms might suffer from metastabilty. In this article, we address this issue by combining systematic control approaches with the heuristic adaptive metadynamics method. Crucially, we approximate the importance sampling control by a neural network, which makes the algorithm in principle feasible for high-dimensional applications. We can numerically demonstrate in relevant metastable problems that our algorithm is more effective than previous attempts and that only the combination of the two approaches leads to a satisfying convergence and therefore to an efficient sampling in certain metastable settings. [ABSTRACT FROM AUTHOR]

This paper presents a comprehensive framework addressing optimal nonlinear analysis and feedback control synthesis for nonlinear stochastic dynamical systems. The focus lies on establishing connections between stochastic Lyapunov theory and stochastic Hamilton–Jacobi–Bellman theory within a unified perspective. We demonstrate that the closed-loop nonlinear system's asymptotic stability in probability is ensured through a Lyapunov function, identified as the solution to the steady-state form of the stochastic Hamilton–Jacobi–Bellman equation. This dual assurance guarantees both stochastic stability and optimality. Additionally, optimal feedback controllers for affine nonlinear systems are developed using an inverse optimality framework tailored to the stochastic stabilization problem. Furthermore, the paper derives stability margins for optimal and inverse optimal stochastic feedback regulators. Gain, sector, and disk margin guarantees are established for nonlinear stochastic dynamical systems controlled by nonlinear optimal and inverse optimal Hamilton–Jacobi–Bellman controllers. [ABSTRACT FROM AUTHOR]

Summary: This paper deals with the second‐order necessary optimality conditions for discrete‐time stochastic optimal control problems under weakened convexity assumptions. Using a special variation of the control, and by virtue of a new discrete‐time backward stochastic equation, we establish a more general and constructive first‐order necessary optimality condition in the form of a global stochastic maximum principle. Moreover, by introducing a new discrete‐time backward stochastic matrix equation, the second‐order multipoint necessary optimality conditions of singular controls are derived, which covers and improves the classical second‐order necessary optimality conditions of discrete‐time stochastic systems. [ABSTRACT FROM AUTHOR]

In this paper, the stochastic control system under partial observation for singular optimal controls in the classical sense is studied. Note that both the system and the observation contain noise, in which the control variable is allowed to enter into all the coefficient terms. By adding some regularity assumptions, the pointwise second-order maximum principle under partial observation can be obtained by Malliavin calculus. Eventually, an example is given to demonstrate how to find the optimal control. [ABSTRACT FROM AUTHOR]

Defined benefit (DB) pension plans are a primary type of pension schemes with the sponsor assuming most of the risks. Longevity-indexed bonds have been used to hedge or transfer risks in pension plans. Our objective is to study an aggregated DB pension plan's optimal risk management problem focusing on minimizing the solvency risk over a finite time horizon and to investigate the investment strategies in a market, comprising a longevity-indexed bond and a risk-free asset, under stochastic nominal interest rates. Using the dynamic programming technique in the stochastic control problem, we obtain the closed-form optimal investment strategy by solving the corresponding Hamilton–Jacobi–Bellman (HJB) equation. In addition, a comparative analysis implicates that longevity-indexed bonds significantly reduce solvency risk compared to zero-coupon bonds, offering a strategic advantage in pension fund management. Besides the closed-form solution and the comparative study, another novelty of this study is the extension of actuarial liability (AL) and normal cost (NC) definitions, and we introduce the risk neutral valuation of liabilities in DB pension scheme with the consideration of mortality rate. [ABSTRACT FROM AUTHOR]

We study the optimal consumption, investment and lifeinsurance purchase and selection strategies for a wage-earner with an uncertain lifetime with access to a financial market comprised of one risk-free security and one risky-asset whose prices evolve according to linear diffusions modulated by a continuous-time stochastic process determined by an additional diffusive nonlinear stochastic differential equation. The process modulating the linear diffusions may be regarded as an indicator describing the state of the economy in a given instant of time. Additionally, we allow the Brownian motions driving each of these equations to be correlated. The lifeinsurance market under consideration herein consists of a fixed number of providers offering pairwise distinct contracts. We use dynamic programming techniques to characterize the solutions to the problem described above for a general family of utility functions, studying the case of discounted constant relative risk aversion utilities with more detail. [ABSTRACT FROM AUTHOR]

This paper is concerned with an optimal control problem for a mean-field linear stochastic differential equation with a quadratic functional in the infinite time horizon. Under suitable conditions, including the stabilizability, the (strong) exponential, integral, and mean-square turnpike properties for the optimal pair are established. The keys are to correctly formulate the corresponding static optimization problem and find the equations determining the correction processes. These have revealed the main feature of the stochastic problems which are significantly different from the deterministic version of the theory. [ABSTRACT FROM AUTHOR]

We consider a class of stochastic optimal control problems with partial observation, and study their approximation by discrete-time control problems. We establish a convergence result by using the weak convergence technique of Kushner and Dupuis [Numerical Methods for Stochastic Control Problems in Continuous Time, Springer, New York], together with the notion of relaxed control rule introduced by El Karoui, Huú Nguyen and Jeanblanc-Picqué [SIAM J. Control Optim., 26 (1988), pp. 1025--1061]. In particular, with a well chosen discrete-time control system, we obtain a first implementable numerical algorithm (with convergence) for the partially observed control problem. Moreover, our discrete-time approximation result would open the door to study convergence of more general numerical approximation methods, such as machine learning based methods. Finally, we illustrate our convergence result by numerical experiments on a partially observed control problem in a linear quadratic setting. [ABSTRACT FROM AUTHOR]

This paper focuses on the application of meta-reinforcement learning to the robust design of low-thrust interplanetary trajectories in the presence of multiple uncertainties. A closed-loop control policy is used to optimally steer the spacecraft to a final target state despite the considered perturbations. The control policy is approximated by a deep recurrent neural network, trained by policy-gradient reinforcement learning on a collection of environments featuring mixed sources of uncertainty, namely dynamic uncertainty and control execution errors. The recurrent network is able to build an internal representation of the distribution of environments, thus better adapting the control to the different stochastic scenarios. The results in terms of optimality, constraint handling, and robustness on a fuel-optimal low-thrust transfer between Earth and Mars are compared with those obtained via a traditional reinforcement learning approach based on a feed-forward neural network. • In interplanetary space missions, the spacecraft trajectory is affected by multiple uncertainties. • Meta-reinforcement learning can be applied seamlessly to any uncertainty and dynamic model. • A recurrent neural network is used as a history-dependent closed-loop control policy. • The network is trained on a low-thrust transfer featuring dynamic uncertainties and control execution errors. • Meta-reinforcement learning shows improved performance and robustness compared to standard reinforcement learning. [ABSTRACT FROM AUTHOR]

In this paper, a computational method based on parameterizing state and control variables is presented for solving Stochastic Optimal Control (SOC) problems. By using Chebyshev wavelets with unknown coefficients, state and control variables are parameterized, and then a stochastic optimal control problem is converted to a stochastic optimization problem. The expected cost functional of the resulting stochastic optimization problem is approximated by sample average approximation thereby the problem can be solved by optimization methods more easily. For facilitating and guaranteeing convergence of the presented method, a new theorem is proved. Finally, the proposed method is implemented based on a newly designed algorithm for solving one of the well-known problems in mathematical finance, the Merton portfolio allocation problem in finite horizon. The simulation results illustrate the improvement of the constructed portfolio return. [ABSTRACT FROM AUTHOR]

In this paper, we consider a joint dynamic pricing and production policy for a stochastic inventory system with perishable products. The demand is stochastic and dependent on the price and the level of the on-hand inventory. Combined dynamic pricing and production control, a stochastic dynamic optimization model that maximizes the total discounted profit is built. Applying the stochastic optimal control theory, we formulate the problem of finding the optimal joint dynamic pricing and production schedule as the problem of solving a Hamilton-Jacobi-Bellman (HJB) equation. By solving the HJB equation, analytical solutions for optimal pricing and production rate are obtained. In addition, a numerical example is given to illustrate the validness of the theoretical results and sensitivity analysis on major system parameters is carried out. [ABSTRACT FROM AUTHOR]

The article discusses the advancement of the G-stochastic maximum principle (G-SMP) from a risk-neutral framework to a risk-sensitive one. The study introduces an auxiliary process to address risk-sensitive performance costs within the G-expectation framework and establishes the correlation between the G-expected exponential utility and the G-quadratic backward stochastic differential equation. The G-adjoint process is simplified, and the necessary optimality conditions for the model are explained. The article presents two examples to illustrate the main findings. [Extracted from the article]

We study a stochastic control problem of mean‐field controlled stochastic differential systems driven by a fractional Brownian motion with Hurst parameter H∈(1/2,1)$$ H\in \left(1/2,1\right) $$. As a necessary condition of the optimal control we obtain a stochastic maximum principle. The associated adjoint mean‐field backward stochastic differential equation driven by a fractional Brownian motion and a classical Brownian motion. Applying the stochastic maximum principle to a mean‐field stochastic linear quadratic problem, we obtain the optimal control and prove that the necessary condition for the optimality of an admissible control is also sufficient under certain assumptions. [ABSTRACT FROM AUTHOR]

A class of fully coupled forward–backward stochastic difference equations with delay (FBSDDEs) over infinite horizon are considered in this article. By establishing a non‐homogeneous explicit relation between the forward and backward equations in terms of Riccati‐like difference equations, we derive the unique solution to the FBSDDEs under certain conditions. Then, we deduce that the FBSDDEs are solvable if and only if the corresponding stochastic delayed system is β$$ \beta $$‐degree open‐loop mean‐square exponentially stabilizable. Finally, as an application, the FBSDDEs are employed to demonstrate the maximum principle of the stochastic LQ optimal control problem. [ABSTRACT FROM AUTHOR]

This paper focuses on a stochastic goodwill model involving a single manufacturer and a single retailer in a supply chain, in which both quality level and goodwill level are governed by stochastic differential equations. In both the linear-quadratic Stackelberg game and linear-quadratic cooperative game, explicit expressions are obtained for the optimal quality improvement efforts level of the manufacturer, the optimal advertising efforts level of the retailer as well as the optimal profit of the supply chain. We also derive the expectation and variance values for quality and goodwill levels, respectively. The distribution of incremental profit and subsidy are analysed in the cooperative game via the utility theory. Finally, several numerical experiments are reported to show the influence of decay rates and white noise disturbance of the quality and goodwill levels on the expectation of quality and goodwill levels. [ABSTRACT FROM AUTHOR]

The optimal control of non-instantaneous impulsive second-order stochastic McKean–Vlasov evolution system with Clarke subdifferential and mixed fractional Brownian motion is investigated in this article. The deterministic nonlinear second-order controlled partial differential system is enriched with stochastic perturbations, non-instantaneous impulses, and Clarke subdifferential. In particular, the nonlinearities in the system that rely on the state of the solution are allowed to rely on the corresponding probability distribution of the state. The solvability of the considered system is discussed with the help of stochastic analysis, multivalued analysis, and multivalued fixed point theorem. Further, the existence of optimal control is established with the aid of Balder's theorem. Finally, an example is provided to illustrate the developed theory. [ABSTRACT FROM AUTHOR]

We propose a refined dynamic programming model based on a hidden Markov chain formulation and a nonlinear filtering technique to calculate the optimal debt ratio for public and private sectors for different scenarios. We then conduct the empirical analysis of the U.S. markets in real estate and equities during 1991.Q1 and 2020.Q1, comparing them with the theoretical results. It indicates that U.S. households and governments spent more than they can afford. While households reduced their debt ratio during times of economic distress, the public sector hiked its debt ratio to stimulate the economy. The policy effect took a long time to accumulate, and the outcome was lower than expected to revitalize the economy in time. [ABSTRACT FROM AUTHOR]

On online platforms, goods, services, and content providers, also known as agents, introduce adverse events. The frequency of these events depends on each agent's effort level. In "Efficient Resource Allocation Contracts to Reduce Adverse Events," Liang, Sun, Tang, and Zhang study continuous-time dynamic contracts that utilize resource allocation and monetary transfers to induce agents to exert effort and reduce the arrival rate of adverse events. They devise an iterative algorithm that characterizes and calculates such contracts and specify the profit-maximizing contract for the platform, also known as the principal. In contrast to the single-agent case, in which efficiency is not achievable, they show that efficient and incentive-compatible contracts, which allocate all resources and induce agents to exert constant effort, generally exist with two or more agents. Additionally, they also provide efficient and incentive-compatible dynamic contracts that can be expressed in closed form and are therefore easy to understand and implement in practice. Motivated by the allocation of online visits to product, service, and content suppliers in the platform economy, we consider a dynamic contract design problem in which a principal constantly determines the allocation of a resource (online visits) to multiple agents. Although agents are capable of running the business, they introduce adverse events, the frequency of which depends on each agent's effort level. We study continuous-time dynamic contracts that utilize resource allocation and monetary transfers to induce agents to exert effort and reduce the arrival rate of adverse events. In contrast to the single-agent case, in which efficiency is not achievable, we show that efficient and incentive-compatible contracts, which allocate all resources and induce agents to exert constant effort, generally exist with two or more agents. We devise an iterative algorithm that characterizes and calculates such contracts, and we specify the profit-maximizing contract for the principal. Furthermore, we provide efficient and incentive-compatible dynamic contracts that can be expressed in closed form and are therefore easy to understand and implement in practice. Funding: Y. Liang acknowledges support from the National Key R&D Program of China [Grant 2020AAA0103801] and the National Natural Science Foundation of China [Grant 71872095]. Supplemental Material: The online appendix is available at https://doi.org/10.1287/opre.2022.2322. [ABSTRACT FROM AUTHOR]

To overcome the time-consuming training caused by the sparse reward function in reinforcement learning, an efficient dense reward framework for robust low-thrust trajectory optimization is proposed. The dense reward functions are designed separately for the deterministic and considered uncertain scenarios, including state uncertainties, observation uncertainties and execution uncertainties. For the uncertainties, the dense reward function is designed to diminish the deviation with respect to the isochronous nominal state along the corresponding deterministic optimal trajectory at each step, rendering the reward function no longer sparse and suitable for complex multirevolution problems. In addition, a multistage reward function of the terminal constraints for the rendezvous missions is designed by incorporating some exponential acceleration terms, enabling significant improvement in training efficiency as the terminal errors become low. In addition, a dense reward function for the deterministic scenario is also proposed via the introduction of empirical forbidden zones and an exponential term. The effectiveness and efficiency of the proposed method is demonstrated in a simple Earth-Mars mission and a complex Earth-Venus multirevolution mission. The promising results verify the significant effect of the proposed method in speeding up the process of training an initial incapable agent to an 'expert' while guaranteeing or even improving the performance. [ABSTRACT FROM AUTHOR]

We develop an approach for two-player constraint zero-sum and nonzero-sum stochastic differential games, which are modeled by Markov regime-switching jump-diffusion processes. We provide the relations between a usual stochastic optimal control setting and a Lagrangian method. In this context, we prove corresponding theorems for two different types of constraints, which lead us to find real-valued and stochastic Lagrange multipliers, respectively. Then, we illustrate our results for a nonzero-sum game problem with the stochastic maximum principle technique. Our application is an example of cooperation between a bank and an insurance company, which is a popular, well-known business agreement type called Bancassurance. [ABSTRACT FROM AUTHOR]

BACKGROUND AND OBJECTIVES: In this paper, a life insurance product is designed with the help of stochastic control approach. These products are defined in such a way that in exchange for receiving an amount as insurance premium that is paid at specified times, the insurer undertakes to pay insurance benefits when the insured is alive at the end of the contract. METHODS: This research is an analytical study in terms of developmental-applicative purpose. In the literature of life insurance, there are various products that are not the same in the type of benefit payment and the timing of their implementation. Among these examples, term life insurance, term life insurance, and mixed life insurance can be mentioned. Traditional insurance products with fixed benefits are quickly losing their appeal due to inflationary markets. In this research, it is focused on the design of a life insurance product on the condition of life, which is connected to the investment markets. Stochastic differential calculus models have been used to simulate capital markets assets. All the numerical results of this research have been calculated with the help of Matlab and Maple software. FINDINGS: To achieve the best choice of investment type, with the help of stochastic optimal control tool, the best investment strategy was calculated for a person who has the CRRA utility function and buys this product, so that the most benefits are paid at the end of the contract. To invest in this contract, modeling was done in a non-risky market such as a bank and a risky market such as stocks, which have price jumps. In addition, to model the risk asset, the Merton model, which is a representative of the models with finite activity, was used, and at the end, a comparison was made for several mortality functions. CONCLUSION: The main purpose of this article is investment for the insured who bought this product. In the product designed in this article, the insurer undertakes to pay the premiums received at a guaranteed rate at the end of the contract. Also, the insured will share the profit from the investment based on a certain percentage that is determined at the beginning of each year. The simulations show that the behavior of the optimal consumption rate is the same as the Merton model with the approach that the behavior of full price jumps is transparent in the optimal consumption rate designed in this article. Investment results for several mortality functions are reported in the Numerical Results section. [ABSTRACT FROM AUTHOR]

The Pontryagin type maximum principle and Bellman's dynamic programming principle serve as two of the most important tools in solving optimal control problems. There is a huge literature on the study of relationship between them. The main purpose of this paper is to investigate the relationship between the Pontryagin type maximum principle and the dynamic programming principle for control systems governed by stochastic evolution equations in infinite dimensional space, with the control variables entering into both the drift and the diffusion terms. To do so, we first prove the dynamic programming principle for those systems without employing the martingale solutions. Then we establish the desired relationships in both cases that value function associated is smooth or nonsmooth. For the nonsmooth case, in particular, by employing the relaxed transposition solution, we discover the connection between the superdifferentials and subdifferentials of value function and the first-order and second-order adjoint equations. [ABSTRACT FROM AUTHOR]

This study applies the theory of stochastic control to an environmental project planning to counteract against the sediment starvation problem in river environments. This can be considered as a time‐average inventory problem to time‐discretely control a continuous‐time system driven by a non‐smooth jump process under habit formation disturbing project implementation. The system is modeled such that the sediment storage dynamics are physically consistent with certain experimental results. Further, the habit formation is modeled as simple linear dynamics and serves as a constraint related to the replenishment amount of the sediment. We show that the time‐average control problem is not necessarily ergodic. Consequently, the effective Hamiltonian may become a non‐constant. Thereafter, ratcheting cases as extreme cases of the irreversible habit formation are considered, owing to them being unique exactly solvable non‐ergodic control problems. The optimality equation associated with a regularized and hence well‐defined control problem is verified. Furthermore, a finite difference scheme is examined against the exactly solvable case and then applied to more complicated cases. [ABSTRACT FROM AUTHOR]

This paper studies the optimal investment choices and benefit adjustment strategy for a target benefit plan (TBP) with stochastic mortality force. The pension fund is invested in a risk-free asset, a stock, and a longevity-linked asset, as a derivative of the mortality force. Using the stochastic optimal control approach, we obtain closed-form solutions for optimal portfolio choices and benefit adjustment strategy in the financial market with or without the longevity-linked asset, which minimize the combination of benefit gap from the target level and pension plan's discontinuity risk. Numerical analysis is provided to show the effects of parameters on the optimal strategies. [ABSTRACT FROM AUTHOR]

This paper studies the numerical algorithm of stochastic control problems in investment optimization. Investors choose the optimal investment to maximize the expected return under uncertainty. The optimality condition, the Hamilton–Jacobi–Bellman (HJB) equation, satisfied by the value function and obtained by the dynamic programming method, is a partial differential equation coupled with optimization. One of the major computational difficulties is the irregular boundary conditions presented in the HJB equation. In this paper, two mesh-free algorithms are proposed to solve two different cases of HJB equations with regular and irregular boundary conditions. The model of optimal investment under uncertainty developed by Abel is used to study the efficacy of the proposed algorithms. Extensive numerical studies are conducted to test the impact of the key parameters on the numerical efficacy. By comparing the numerical solution with the exact solution, the proposed numerical algorithms are validated. [ABSTRACT FROM AUTHOR]

In this paper, we solve an optimal reinsurance problem in the mathematical finance area. We assume that the surplus process of the insurance company follows a controlled diffusion process and the constant interest rate is involved in the financial model. During the whole optimization period, the company has a choice to buy reinsurance contract and decide the reinsurance retention level. Meanwhile, the bankruptcy at the terminal time is not allowed. The aim of the optimization problem is to minimize the distance between the terminal wealth and a given goal by controlling the reinsurance proportion. Using the stochastic control theory, we derive the Hamilton-Jacobi-Bellman equation for the optimization problem. Via adopting the technique of changing variable as well as the dual transformation, an explicit solution of the value function and the optimal policy are shown. Finally, several numerical examples are shown, from which we find several main factors that affect the optimal reinsurance policy. [ABSTRACT FROM AUTHOR]

The optimization and control of spacecraft trajectories in presence of uncertainties and unmodeled disturbances represents a challenging problem. Future missions would benefit from robust control techniques to compute their trajectory in a complex, nonlinear environment and in presence of stochasticity. In this work the Stochastic Optimal Control problem in the Circular Restricted Three Body Problem is addressed and a method is proposed based on Nonlinear Model Predictive control (NMPC). This method presents the challenge of solving a nonlinear optimization problem on-board. A Quantum-inspired meta-heuristic optimization algorithm is tested in this context: the Diffusion Monte Carlo (DMC) method is based on a numerical techniques used in Quantum Mechanics, adapted to solve optimization problems. The NMPC with DMC is implemented, taking into account an estimate of the computational time required through the introduction of delayed actuation of the control. Two applications are presented featuring transfer trajectories between planar periodic orbits in the vicinity of the Moon; the proposed method proved able to successfully generate trajectories when disturbance accelerations and uncertain initial conditions are present. • The Circular Restricted Three Body Problem with uncertainties is considered. • A NMPC is used to drive the spacecraft to the desired orbit despite the uncertainty. • A Quantum-inspired meta-heuristic optimizer is used in the MPC. • Applications are presented regarding transfers between Lunar periodic orbits. [ABSTRACT FROM AUTHOR]

To address the issue of inaccurate distributions in discrete-time stochastic systems, a minimax linear quadratic control method using theWasserstein metric is proposed. Our method aims to construct a control policy that is robust against errors in an empirical distribution of underlying uncertainty by adopting an adversary that selects the worst-case distribution at each time. The opponent receives a Wasserstein penalty proportional to the amount of deviation from the empirical distribution. As a tractable solution, a closed-form expression of the optimal policy pair is derived using a Riccati equation. We identify nontrivial stabilizability and observability conditions under which the Riccati recursion converges to the unique positive semidefinite solution of an algebraic Riccati equation. Our method is shown to possess several salient features, including closed-loop stability, a guaranteed-cost property, and a probabilistic out-of-sample performance guarantee. [ABSTRACT FROM AUTHOR]

This paper deals with the optimal control of grid‐connected Battery Energy Storage Systems (BESSs) operating for energy arbitrage. An important issue is that BESSs degrade over time, according to their use, and thus they are usable only for a limited number of cycles. Therefore, the time horizon of the optimization depends on the actual operation of the BESS. We focus on Li‐ion batteries and use an empirical model to describe battery degradation. The BESS model includes an equivalent circuit for the battery and a simplified model for the power converter. In order to model the energy price variations, we use a linear stochastic model that includes the effect of the time‐of‐the‐day. The problem of maximizing the revenues obtained over the BESS lifetime is formulated as a stochastic optimal control problem with a long, operation‐dependent time horizon. First, we divide this problem into a finite set of subproblems, such that for each one of them, the State of Health (SoH) of the battery is approximately constant. Next, we reformulate approximately every subproblem into the minimization of the ratio of two long‐time average‐cost criteria and use a value‐iteration‐type algorithm to derive the optimal policy. Finally, we present some numerical results and investigate the effects of the energy loss parameters, degradation parameters, and price dynamics on the optimal policy. [ABSTRACT FROM AUTHOR]

A stochastic epidemic model with random noise transmission is taken into account, describing the dynamics of the measles viral infection. The basic reproductive number is calculated corresponding to the stochastic model. It is determined that, given initial positive data, the model has bounded, unique, and positive solution. Additionally, utilizing stochastic Lyapunov functional theory, we study the extinction of the disease. Stationary distribution and extinction of the infection are examined by providing sufficient conditions. We employed optimal control principles and examined stochastic control systems to regulate the transmission of the virus using environmental factors. Graphical representations have been offered for simplicity of comprehending in order to further verify the acquired analytical findings. [ABSTRACT FROM AUTHOR]

We consider the diffusive limit of a typical pure-jump Markovian control problem as the intensity of the driving Poisson process tends to infinity. We show that the convergence speed is provided by the Hölder exponent of the Hessian of the limit problem, and explain how correction terms can be constructed. This provides an alternative efficient method for the numerical approximation of the optimal control of a pure-jump problem in situations with very high intensity of jumps. We illustrate this approach in the context of a display advertising auction problem. [ABSTRACT FROM AUTHOR]

This paper is concerned with a class of mean-feld type coupled forward-backward stochastic diferential equations (MF-FBSDEs), in which the coupling appears in integral terms, terminal terms, and initial terms. Inspired by various mean-feld type linear-quadratic (MF-LQ) optimal control problems, we propose a type of randomized domination-monotonicity conditions, under which and the usual Lipschitz condition we obtain a well-posedness result on MF-FBSDEs in the sense of square integrability including the unique solvability, an estimate of the solution, and the related continuous dependence property of the solution on the coefcients. The result of MF- FBSDEs in turn extends MF-LQ problems in the literature to a general situation where the initial states or the terminal states are also controlled at the same time, and gives explicit expressions of the related unique optimal controls. [ABSTRACT FROM AUTHOR]

This paper considers an optimal control problem for a time-invariant linear stochastic system with discrete time, scalar unbounded control, additive noise, and a probabilistic criterion for retaining its trajectories in a given neighborhood of zero. We use dynamic programming and two-sided Bellman function estimates to derive analytical expressions for the optimal control at two time steps and a suboptimal control on any control horizon. The effectiveness of these controls is illustrated on a numerical example. [ABSTRACT FROM AUTHOR]

We study singular perturbations of a class of two-scale stochastic control systems with unbounded data. The assumptions are designed to cover some relaxation problems for deep neural networks. We construct effective Hamiltonian and initial data and prove the convergence of the value function to the solution of a limit (effective) Cauchy problem for a parabolic equation of HJB type. We use methods of probability, viscosity solutions and homogenization. [ABSTRACT FROM AUTHOR]

We study an exit contract design problem, where one provides a universal exit contract to multiple heterogeneous agents, with which each agent chooses an optimal (exit) stopping time. The problem consists in optimizing the universal exit contract w.r.t. some criterion depending on the contract as well as the agents' exit times. Under a technical monotonicity condition, and by using Bank-El Karoui's representation of stochastic processes, we are able to transform the initial contract optimization problem into an optimal control problem. The latter is also equivalent to an optimal multiple stopping problem and the existence of the optimal contract is proved. We next show that the problem in the continuous-time setting can be approximated by a sequence of discrete-time ones, which would induce a natural numerical approximation method. We finally discuss the optimization problem over the class of all Markovian and/or continuous exit contracts. [ABSTRACT FROM AUTHOR]

This article consists of a detailed and novel stochastic optimal control analysis of a coupled non-linear dynamical system. The state equations are modelled as an additional food-provided prey–predator system with Holling type III functional response for predator and intra-specific competition among predators. We first discuss the optimal control problem as a Lagrangian problem with a linear quadratic control. Second, we consider an optimal control problem in the time-optimal control setting. We initially establish the existence of optimal controls for both these problems and later characterize these optimal controls using the Stochastic maximum principle. Further numerical simulations are performed based on stochastic forward-backward sweep methods for realizing the theoretical findings. The results obtained in these optimal control problems are discussed in the context of biological conservation and pest management. [ABSTRACT FROM AUTHOR]