Search

We study an optimal control problem for a stochastic model of tumour growth with drug application. This model consists of three stochastic hyperbolic equations describing the evolution of tumour cells. It also includes two stochastic parabolic equations describing the diffusions of nutrient and drug concentrations. Since all systems are subject to many uncertainties, we have added stochastic terms to the deterministic model to consider the random perturbations. Then, we have added control variables to the model according to the medical concepts to control the concentrations of drug and nutrient. In the optimal control problem, we have defined the stochastic and deterministic cost functions and we have proved the problems have unique optimal controls. For deriving the necessary conditions for optimal control variables, the stochastic adjoint equations are derived. We have proved the stochastic model of tumour growth and the stochastic adjoint equations have unique solutions. For proving the theoretical results, we have used a change of variable which changes the stochastic model and adjoint equations (a.s.) to deterministic equations. Then we have employed the techniques used for deterministic ones to prove the existence and uniqueness of optimal control. [ABSTRACT FROM AUTHOR]

Problem definition: More than 99% of the new power generation capacity to be installed in the United States from 2023 to 2050 will be powered by wind, solar, and natural gas. Additionally, large-scale battery systems are planned to support power systems. It is paramount for policymakers and electric utilities to deepen the understanding of the operational and investment relations among renewable, flexible (natural gas-powered), and storage capacities. In this paper, we optimize both the joint operations and investment mix of these three types of resources, examining whether they act as investment substitutes or complements. Methodology/results: Using stochastic control theory, we identify and prove the structure of the optimal storage control policy, from which we determine various pairs of charging and discharging operations. We find that whether storage complements or substitutes other resources hinges on the operational pairs involved and whether executing these pairs is constrained by charging or discharging. Through extensive numerical analysis using data from a Florida utility, government agencies, and industry reports, we demonstrate how storage operations drive the investment relations among renewable, flexible, and storage capacities. Managerial implications: Storage and renewables substitute each other in meeting peak demand; storage complements renewables by storing surplus renewable output; renewables complement storage by compressing peak periods, facilitating peak shaving and displacement of flexible capacity. These substitution and complementary effects often coexist, and the dominant effect can alternate as costs change. A thorough understanding of these relations at both operational and investment levels empowers decision makers to optimize energy infrastructure investments and operations, thereby unlocking their full potential. Funding: This research was supported in part by Lilly Endowment, Inc., through its support for the Indiana University Pervasive Technology Institute. This research was also supported by Kelley School of Business, Indiana University, and Haslam College of Business, University of Tennessee. O. Q. Wu thanks Grant Thornton for their generous support. Supplemental Material: The online appendix is available at https://doi.org/10.1287/msom.2023.0068. [ABSTRACT FROM AUTHOR]

We consider a continuous-time model of optimal consumption and pension insurance for a consumer with an uncertain lifetime. In the model, the consumer earns a stochastic wage income during her working life and optimally allocates her income between personal consumption, pension insurance, and securities with a deterministic dynamic return. Due to the weak development of the stock market in developing countries, employees' income comes mainly from wages and interest on savings from banks, that are discussed in this paper. The consumer's utility and bequest functions are constant absolute risk aversion (CARA). By characterizing the optimality condition of the consumer's problem using the Hamilton-Jacobi-Bellman equation, we find the optimal consumption and pension insurance as a function of wealth in closed form. We consider an application of the model while estimating its key elements using real-life data on age-specific population size, labor income, and interest rates. We show that as the absolute risk aversion for consumption increases, consumption and wealth move in the opposite direction. We also present a novel finding that wealth and consumption can be negatively related across consumers with different levels of consumption risk aversion. [ABSTRACT FROM AUTHOR]

This paper is concerned with the turnpike property for a class of stochastic linear-quadratic (LQ, for short) optimal control problems with periodic coefficients. The stability and stabilizability of the control system are studied, followed by the discussion of the existence and uniqueness of periodic solutions. A deterministic periodic LQ problem is introduced and solved, whose optimal pair, together with a pair of correction processes, serves as the turnpike limit of the stochastic problem. It is shown that the turnpike limit is periodic in the distribution sense. In the special case of constant coefficients, the turnpike limit turns out to have stationary distributions, with the expectation being the solution to a static optimization problem. [ABSTRACT FROM AUTHOR]

This paper presents convergence analysis of a novel data-driven feedback control algorithm designed for generating online controls based on partial noisy observational data. The algorithm comprises a particle filter-enabled state estimation component, estimating the controlled system's state via indirect observations, alongside an efficient stochastic maximum principle-type optimal control solver. By integrating weak convergence techniques for the particle filter with convergence analysis for the stochastic maximum principle control solver, we derive a weak convergence result for the optimization procedure in search of optimal data-driven feedback control. Numerical experiments are performed to validate the theoretical findings. [ABSTRACT FROM AUTHOR]

The sandwich plate (SP) with supported mass can model structural systems such as platform or floor with installed vibration-sensitive apparatus under random loading. The stochastic optimal control (in time domain) of periodic (in space) viscoelastomer (VE) SP with supported mass subjected to random excitation is an important research subject, which can fully use VE controllability, but it is a challenging problem on optimal bounded parametric control (OBPC). In this paper, a stochastic OBPC for periodic VESP with supported mass subjected to random base loading is proposed according to the stochastic dynamical programming (SDP) principle. Response-reduction capability using the proposed OBPC is studied to demonstrate further control effectiveness of periodic SP via SDP. Controllable VE core modulus of SP is distributed periodically in space. Differential equations for coupling vibration of periodic SP with supported mass are derived and transformed into multi-dimensional system equations with parameters as nonlinear functions of bounded control. The OBPC problem is established by the system equations and performance index with bound constraint. Then, an SDP equation is derived according to the SDP principle. The OBPC law is obtained from the SDP equation under bound constraint. Optimally controlled responses are calculated and compared with passively controlled responses to evaluate control effectiveness. Numerical results on responses and statistics of SP via the proposed OBPC show further remarkable control effectiveness. [ABSTRACT FROM AUTHOR]

The cyber risk insurance market is at a nascent stage of its development, even as the magnitude of cyber losses is significant and the rate of cyber loss events is increasing. Existing cyber risk insurance products as well as academic studies have been focusing on classifying cyber loss events and developing models of these events, but little attention has been paid to proposing insurance risk transfer strategies that incentivise mitigation of cyber loss through adjusting the premium of the risk transfer product. To address this important gap, we develop a Bonus-Malus model for cyber risk insurance. Specifically, we propose a mathematical model of cyber risk insurance and cybersecurity provisioning supported with an efficient numerical algorithm based on dynamic programming. Through a numerical experiment, we demonstrate how a properly designed cyber risk insurance contract with a Bonus-Malus system can resolve the issue of moral hazard and benefit the insurer. [ABSTRACT FROM AUTHOR]

This paper deals with finite‐horizon minimum‐variance and covariance steering problems subject to constraints. The goal of the minimum variance problem is to steer the state mean of an uncertain system to a prescribed vector while minimizing the trace of its terminal state covariance whereas the goal in the covariance steering problem is to steer the covariance matrix of the terminal state to a prescribed positive definite matrix. The paper proposes a solution approach that relies on a stochastic version of the affine disturbance feedback control parametrization. In this control policy parametrization, the control input at each stage is expressed as an affine function of the history of disturbances that have acted upon the system. It is shown that this particular parametrization reduces the stochastic optimal control problems considered in this paper into tractable convex programs or difference of convex functions programs with essentially the same decision variables. In addition, the paper proposes a variation of this control parametrization that relies on truncated histories of past disturbances, which allows for sub‐optimal controllers to be designed that strike a balance between performance and computational cost. The suboptimality of the truncated policies is formally analyzed and closed form expressions are provided for the performance loss due to the use of the truncation scheme. Finally, the paper concludes with a comparative analysis of the truncated versions of the proposed policy parametrization and other standard policy parametrizations through numerical simulations. [ABSTRACT FROM AUTHOR]

This paper addresses the problem of joint production and preventive maintenance (PM) planning of a deteriorating manufacturing system generating greenhouse gas (GHG) emissions. The system is composed of a deteriorating machine, subject to random failures and repairs evolving in a dynamic and stochastic context. The main objective is to develop control policies that minimize the sum of backlog, inventory, maintenance, and emission costs. The stochastic optimal control theory based on the dynamic programming approach is used to obtain the optimality conditions and the optimal control policies, which are determined using numerical methods. Sensitivity analyses are provided to depict and validate the obtained structure of the production and PM policies characterized by multiple thresholds that jointly regulate the production and PM rates with the age, emissions, and inventory levels. Furthermore, we compared the performance of the obtained control policies with that of the most relevant policies found in the literature and showed their superiority by considerable cost savings. Finally, the proposal's implementation is provided to equip managers of the considered manufacturing system with an effective and robust decision-support tool. [ABSTRACT FROM AUTHOR]

We consider a singularly perturbed system of stochastic differential equations proposed by Chaudhari et al. (Res. Math. Sci. 2018) to approximate the entropic gradient descent in the optimization of deep neural networks via homogenization. We embed it in a much larger class of two-scale stochastic control problems and rely on convergence results for Hamilton--Jacobi--Bellman equations with unbounded data proved recently by ourselves (ESAIM Control Optim. Calc. Var. 2023). We show that the limit of the value functions is itself the value function of an effective control problem with extended controls and that the trajectories of the perturbed system converge in a suitable sense to the trajectories of the limiting effective control system. These rigorous results improve the understanding of the convergence of the algorithms used by Chaudhari et al., as well as of their possible extensions where some tuning parameters are modeled as dynamic controls. [ABSTRACT FROM AUTHOR]

The fashion apparel industry is facing an increasingly growing demand, compounded by the short sales lifecycle and strong seasonality of clothing, posing significant challenges to inventory management in the retail sector. Despite some retailers like Uniqlo and Zara implementing inventory management and dynamic pricing strategies, challenges persist due to the dynamic nature of fashion trends and the stochastic factors affecting inventory. To address these issues, we construct a mathematical model based on the mathematical expression of the deterministic fashion level function, where the geometric Brownian motion, widely applied in finance, is initially utilized in the stochastic fashion level function. Drawing on research findings from deteriorating inventory management and stochastic optimization, we investigate the fluctuation of inventory levels, optimal dynamic pricing, optimal production rates, and profits—four crucial indicators—via Pontryagin's maximum principle. Analytical solutions are derived, and the numerical simulation is provided to verify and compare the proposed model with deterministic fashion level function models. The model emphasizes the importance of considering stochastic factors in decision-making processes and provides insights to enhance profitability, inventory management, and sustainable consumption in the fashion product industry. [ABSTRACT FROM AUTHOR]

Information drives trading and hence affects price dynamics. We study how an informed trader optimally trades, how multiple traders compete with each other, and how their competition affects price dynamics as well as their inventories in the linear price impact model framework. We formulate the informed trading problem as a stochastic optimal control problem and a stochastic differential game in different cases. By virtue of different methods, problems are solved explicitly in all cases. In the case of a single trader, the risk-averse informed trader trades less in the beginning and then speeds up to reduce exposure to the volatility of dynamic information. In addition, the risk-averse trader reveals information more slowly and thus contributes less to price efficiency. When multiple risk-neutral traders compete with information, regardless of the competition structure, they tend to trade more rapidly in the early stage to fill orders at a good price. Competition and the presence of a leader help promote price efficiency, and competition also leads to the empirically observed mean-reverting behavior of price dynamics and order flows. Furthermore, among risk-averse informed traders, inventory management triggers predatory trading and active information leakage during informed trading, and the positions of traders with the same level of risk aversion tend to be identical and mean-reverting to zero under multiple rounds of competition. [ABSTRACT FROM AUTHOR]

We present a two-country model (North and South) that describes the phenomenon of offshoring and reshoring. The model is a continuous time-controlled Markov chain with binary states. The main trade-off involves production costs and transaction costs between one country and another. In the first part of this paper, we identify the key parameters of the model: the difference in unit production costs between the two countries considered, the marginal cost of transitioning between countries, and the incentive paid by the North country to all companies that have not relocated at the end of the planning interval. The final goal of our paper is to understand how national tax incentives can influence this process. [ABSTRACT FROM AUTHOR]

We investigate a stochastic differential game in which a major player has private information (the knowledge of a random variable), which she discloses through her control to a population of small players playing in a Nash mean field game equilibrium. The major player's cost depends on the distribution of the population, while the cost of the population depends on the random variable known by the major player. We show that the game has a relaxed solution and that the optimal control of the major player is approximatively optimal in games with a large but finite number of small players. [ABSTRACT FROM AUTHOR]

In this manuscript we consider a class of optimal control problems of stochastic differential delay equations. First, we rewrite the problem in a suitable infinite-dimensional Hilbert space. Then, using the dynamic programming approach, we characterize the value function of the problem as the unique viscosity solution of the associated infinite-dimensional Hamilton-Jacobi-Bellman equation. Finally, we prove a C1,α -partial regularity of the value function. We apply these results to path dependent financial and economic problems (Merton-like portfolio problem and optimal advertising). [ABSTRACT FROM AUTHOR]

In this paper we explore optimal liquidation in a market populated by a number of heterogeneous market makers that have limited inventory-carrying and risk-bearing capacity. We derive a reduced form model for the dynamics of their aggregated inventory considering a proper scaling limit. The resulting price impact profile is shown to depend on the characteristics and relative importance of their inventories. The model is flexible enough to reproduce the empirically documented power law behavior of the price impact function. For any choice of the market makers characteristics, optimal execution within this modeling approach can be recast as a linear-quadratic stochastic control problem. The value function and the associated optimal trading rate can be obtained semi-explicitly subject to solving a differential matrix Riccati equation. Numerical simulations are conducted to illustrate the performance of the resulting optimal liquidation strategy in relation to standard benchmarks. Remarkably, they show that the increase in performance is determined by a substantial reduction of higher order moment risk. [ABSTRACT FROM AUTHOR]

In this paper, the stochastic optimal control of a tri-stable energy harvester (TEH) under a standard rectifier circuit driven by correlated colored noises is considered. From the view of physical intuition, the control force is directly separated into conservative component and dissipative one. Then, the stationary probability density function (SPDF) and the DC power of the controlled electromechanical coupled TEH can be derived by using the stochastic averaging based on energy-dependent frequency. The weighted combination of mean DC power and rectification efficiency is regarded as a performance index to transform the stochastic optimal control problem into an extremum problem of a multivariable function. The stochastic direct optimal control strategy developed in this paper avoids the trouble of dealing with complex differential equations. In contrast with the uncontrolled case, the TEH under direct optimal control achieves significant optimization of harvesting performance. The cross-correlation between the additive and multiplicative colored noises can break the symmetry of SPDF to induce random transition. The power conversion efficiency of the harvesting system can be optimized by choosing the appropriate noise intensities. [ABSTRACT FROM AUTHOR]

Navigating the Complex Web of Incentives in Asset Management: A Study on Investor, Partner, and Manager Dynamics In a recent study titled "Dynamic Contracting in Asset Management Under the Investor-Partner-Manager Relationship," the researchers delve into the intricate world of incentives in asset management. The focus is on the complex interplay of actions and relationships among three key players: an investor, an investment company partner, and a fund manager. This study uniquely explores the challenges of incentive contracts in an environment where the actions of these individuals are not fully observable to each other. It introduces a hierarchical contracting framework to understand the dynamics under the hidden actions. For instance, the research reveals how the optimal action of the manager is influenced by the cost of the partner's actions and extends the analysis to scenarios involving multiple managers. This work sheds light on the often hidden intricacies of asset management, providing valuable perspectives for the different players' collaborative efforts for optimal outcomes in this industry. We study incentive contracts in asset management business under dynamic actions and relationships between an investor, a partner of an investment company, and a fund manager of the company. Both the manager and the partner exert costly effort to manage the investments. The investor cannot perfectly observe the partner's and manager's actions, and similarly, the partner cannot perfectly observe the manager's actions. Thus, we consider a hierarchical contracting framework under hidden efforts, where the investor contracts with the partner and the partner contracts with the manager. We show how the actions of the participants and the costs of their actions interact. For instance, the optimal effort of the manager falls in the effort cost of the partner. We extend the model to a case with an investor, a partner, and multiple managers. In this case, each manager's effort rises in the effectiveness of the managers' cooperation and falls in the other managers' effort cost. Funding: J. Keppo acknowledges funding support from the Singapore Ministry of Education [Grant R-252-000-A08-112] and from the National Research Foundation (Singapore) and Agency for Science, Technology and Research (Singapore) Industry Alignment Fund–Industry Collaboration Projects [Grant I2001E0059]. N. Touzi acknowledges funding from the Chaire Finance and Sustainable development, Risk Foundation, Louis Bachelier Institute. R. Zuo acknowledges funding support from Guangzhou Municipal Science and Technology Project [2023A03J0163 and 2024A04J4546] and the Guangzhou-HKUST(GZ) joint funding program [2024A03J0630]. Supplemental Material: The online appendix is available at https://doi.org/10.1287/opre.2021.0031. [ABSTRACT FROM AUTHOR]

We consider a continuous-time model of optimal consumption and pension insurance for a consumer with an uncertain lifetime. In the model, the consumer earns a stochastic wage income during her working life and optimally allocates her income between personal consumption, pension insurance, and securities with a deterministic dynamic return. Due to the weak development of the stock market in developing countries, employees' income comes mainly from wages and interest on savings from banks, that are discussed in this paper. The consumer's utility and bequest functions are constant absolute risk aversion (CARA). By characterizing the optimality condition of the consumer's problem using the Hamilton-Jacobi-Bellman equation, we find the optimal consumption and pension insurance as a function of wealth in closed form. We consider an application of the model while estimating its key elements using real-life data on age-specific population size, labor income, and interest rates. We show that as the absolute risk aversion for consumption increases, consumption and wealth move in the opposite direction. We also present a novel finding that wealth and consumption can be negatively related across consumers with different levels of consumption risk aversion.

In this paper, a computational method based on parameterizing state and control variables is presented for solving Stochastic Optimal Control (SOC) problems. By using Chebyshev wavelets with unknown coefficients, state and control variables are parameterized, and then a stochastic optimal control problem is converted to a stochastic optimization problem. The expected cost functional of the resulting stochastic optimization problem is approximated by sample average approximation thereby the problem can be solved by optimization methods more easily. For facilitating and guar-anteeing convergence of the presented method, a new theorem is proved. Finally, the proposed method is implemented based on a newly designed algorithm for solving one of the well-known problems in mathematical fi-nance, the Merton portfolio allocation problem in finite horizon. The simu-lation results illustrate the improvement of the constructed portfolio return.

We consider optimal control problems for a system governed by a stochastic differential equation driven by a d-dimensional Brownian motion where both the drift and the diffusion coefficient are controlled. It is well known that without additional convexity conditions the strict control problem does not admit an optimal control. To overcome this difficulty, we consider the relaxed model, in which admissible controls are measure-valued processes and the relaxed state process is governed by a stochastic differential equation driven by a continuous orthogonal martingale measure. This relaxed model admits an optimal control that can be approximated by a sequence of strict controls by the so-called chattering lemma. We establish optimality necessary conditions, in terms of two adjoint processes, extending Peng's maximum principle to relaxed control problems. We show that relaxing the drift and diffusion martingale parts directly as in deterministic control does not lead to a true relaxed model as the obtained controlled dynamics is not continuous in the control variable. [ABSTRACT FROM AUTHOR]

Sampling rare events in metastable dynamical systems is often a computationally expensive task and one needs to resort to enhanced sampling methods such as importance sampling. Since we can formulate the problem of finding optimal importance sampling controls as a stochastic optimization problem, this then brings additional numerical challenges and the convergence of corresponding algorithms might suffer from metastabilty. In this article, we address this issue by combining systematic control approaches with the heuristic adaptive metadynamics method. Crucially, we approximate the importance sampling control by a neural network, which makes the algorithm in principle feasible for high-dimensional applications. We can numerically demonstrate in relevant metastable problems that our algorithm is more effective than previous attempts and that only the combination of the two approaches leads to a satisfying convergence and therefore to an efficient sampling in certain metastable settings. [ABSTRACT FROM AUTHOR]

This paper presents a comprehensive framework addressing optimal nonlinear analysis and feedback control synthesis for nonlinear stochastic dynamical systems. The focus lies on establishing connections between stochastic Lyapunov theory and stochastic Hamilton–Jacobi–Bellman theory within a unified perspective. We demonstrate that the closed-loop nonlinear system's asymptotic stability in probability is ensured through a Lyapunov function, identified as the solution to the steady-state form of the stochastic Hamilton–Jacobi–Bellman equation. This dual assurance guarantees both stochastic stability and optimality. Additionally, optimal feedback controllers for affine nonlinear systems are developed using an inverse optimality framework tailored to the stochastic stabilization problem. Furthermore, the paper derives stability margins for optimal and inverse optimal stochastic feedback regulators. Gain, sector, and disk margin guarantees are established for nonlinear stochastic dynamical systems controlled by nonlinear optimal and inverse optimal Hamilton–Jacobi–Bellman controllers. [ABSTRACT FROM AUTHOR]

Summary: This paper deals with the second‐order necessary optimality conditions for discrete‐time stochastic optimal control problems under weakened convexity assumptions. Using a special variation of the control, and by virtue of a new discrete‐time backward stochastic equation, we establish a more general and constructive first‐order necessary optimality condition in the form of a global stochastic maximum principle. Moreover, by introducing a new discrete‐time backward stochastic matrix equation, the second‐order multipoint necessary optimality conditions of singular controls are derived, which covers and improves the classical second‐order necessary optimality conditions of discrete‐time stochastic systems. [ABSTRACT FROM AUTHOR]

In this paper, the stochastic control system under partial observation for singular optimal controls in the classical sense is studied. Note that both the system and the observation contain noise, in which the control variable is allowed to enter into all the coefficient terms. By adding some regularity assumptions, the pointwise second-order maximum principle under partial observation can be obtained by Malliavin calculus. Eventually, an example is given to demonstrate how to find the optimal control. [ABSTRACT FROM AUTHOR]

Defined benefit (DB) pension plans are a primary type of pension schemes with the sponsor assuming most of the risks. Longevity-indexed bonds have been used to hedge or transfer risks in pension plans. Our objective is to study an aggregated DB pension plan's optimal risk management problem focusing on minimizing the solvency risk over a finite time horizon and to investigate the investment strategies in a market, comprising a longevity-indexed bond and a risk-free asset, under stochastic nominal interest rates. Using the dynamic programming technique in the stochastic control problem, we obtain the closed-form optimal investment strategy by solving the corresponding Hamilton–Jacobi–Bellman (HJB) equation. In addition, a comparative analysis implicates that longevity-indexed bonds significantly reduce solvency risk compared to zero-coupon bonds, offering a strategic advantage in pension fund management. Besides the closed-form solution and the comparative study, another novelty of this study is the extension of actuarial liability (AL) and normal cost (NC) definitions, and we introduce the risk neutral valuation of liabilities in DB pension scheme with the consideration of mortality rate. [ABSTRACT FROM AUTHOR]

We study the optimal consumption, investment and lifeinsurance purchase and selection strategies for a wage-earner with an uncertain lifetime with access to a financial market comprised of one risk-free security and one risky-asset whose prices evolve according to linear diffusions modulated by a continuous-time stochastic process determined by an additional diffusive nonlinear stochastic differential equation. The process modulating the linear diffusions may be regarded as an indicator describing the state of the economy in a given instant of time. Additionally, we allow the Brownian motions driving each of these equations to be correlated. The lifeinsurance market under consideration herein consists of a fixed number of providers offering pairwise distinct contracts. We use dynamic programming techniques to characterize the solutions to the problem described above for a general family of utility functions, studying the case of discounted constant relative risk aversion utilities with more detail. [ABSTRACT FROM AUTHOR]

This paper is concerned with an optimal control problem for a mean-field linear stochastic differential equation with a quadratic functional in the infinite time horizon. Under suitable conditions, including the stabilizability, the (strong) exponential, integral, and mean-square turnpike properties for the optimal pair are established. The keys are to correctly formulate the corresponding static optimization problem and find the equations determining the correction processes. These have revealed the main feature of the stochastic problems which are significantly different from the deterministic version of the theory. [ABSTRACT FROM AUTHOR]

We consider a class of stochastic optimal control problems with partial observation, and study their approximation by discrete-time control problems. We establish a convergence result by using the weak convergence technique of Kushner and Dupuis [Numerical Methods for Stochastic Control Problems in Continuous Time, Springer, New York], together with the notion of relaxed control rule introduced by El Karoui, Huú Nguyen and Jeanblanc-Picqué [SIAM J. Control Optim., 26 (1988), pp. 1025--1061]. In particular, with a well chosen discrete-time control system, we obtain a first implementable numerical algorithm (with convergence) for the partially observed control problem. Moreover, our discrete-time approximation result would open the door to study convergence of more general numerical approximation methods, such as machine learning based methods. Finally, we illustrate our convergence result by numerical experiments on a partially observed control problem in a linear quadratic setting. [ABSTRACT FROM AUTHOR]

This paper focuses on the application of meta-reinforcement learning to the robust design of low-thrust interplanetary trajectories in the presence of multiple uncertainties. A closed-loop control policy is used to optimally steer the spacecraft to a final target state despite the considered perturbations. The control policy is approximated by a deep recurrent neural network, trained by policy-gradient reinforcement learning on a collection of environments featuring mixed sources of uncertainty, namely dynamic uncertainty and control execution errors. The recurrent network is able to build an internal representation of the distribution of environments, thus better adapting the control to the different stochastic scenarios. The results in terms of optimality, constraint handling, and robustness on a fuel-optimal low-thrust transfer between Earth and Mars are compared with those obtained via a traditional reinforcement learning approach based on a feed-forward neural network. • In interplanetary space missions, the spacecraft trajectory is affected by multiple uncertainties. • Meta-reinforcement learning can be applied seamlessly to any uncertainty and dynamic model. • A recurrent neural network is used as a history-dependent closed-loop control policy. • The network is trained on a low-thrust transfer featuring dynamic uncertainties and control execution errors. • Meta-reinforcement learning shows improved performance and robustness compared to standard reinforcement learning. [ABSTRACT FROM AUTHOR]

For a step process X with respect to its natural filtration F, we denote by G the smallest right-continuous filtration containing F and such that another step process H is adapted. We investigate some structural properties of the step process X in G. We show that Z =(X,H) possesses the weak representation property with respect to G. Moreover, in the case H = 1[t,+8), where t is a random time (but not an F-stopping time) satisfying Jacod's absolute continuity hypothesis, we compute the G-predictable compensator vG,X of the jump measure of X. Thanks to our theoretical results on vG,X, we can consider stochastic control problems related to model uncertainty on the intensity measure of X, also in presence of an external risk source modeled by the random time t. [ABSTRACT FROM AUTHOR]

In this paper, a computational method based on parameterizing state and control variables is presented for solving Stochastic Optimal Control (SOC) problems. By using Chebyshev wavelets with unknown coefficients, state and control variables are parameterized, and then a stochastic optimal control problem is converted to a stochastic optimization problem. The expected cost functional of the resulting stochastic optimization problem is approximated by sample average approximation thereby the problem can be solved by optimization methods more easily. For facilitating and guaranteeing convergence of the presented method, a new theorem is proved. Finally, the proposed method is implemented based on a newly designed algorithm for solving one of the well-known problems in mathematical finance, the Merton portfolio allocation problem in finite horizon. The simulation results illustrate the improvement of the constructed portfolio return. [ABSTRACT FROM AUTHOR]

This paper presents convergence analysis of a novel data-driven feedback control algorithm designed for generating online controls based on partial noisy observational data. The algorithm comprises a particle filter-enabled state estimation component, estimating the controlled system’s state via indirect observations, alongside an efficient stochastic maximum principle-type optimal control solver. By integrating weak convergence techniques for the particle filter with convergence analysis for the stochastic maximum principle control solver, we derive a weak convergence result for the optimization procedure in search of optimal data-driven feedback control. Numerical experiments are performed to validate the theoretical findings.

The sandwich plate (SP) with supported mass can model structural systems such as platform or floor with installed vibration-sensitive apparatus under random loading. The stochastic optimal control (in time domain) of periodic (in space) viscoelastomer (VE) SP with supported mass subjected to random excitation is an important research subject, which can fully use VE controllability, but it is a challenging problem on optimal bounded parametric control (OBPC). In this paper, a stochastic OBPC for periodic VESP with supported mass subjected to random base loading is proposed according to the stochastic dynamical programming (SDP) principle. Response-reduction capability using the proposed OBPC is studied to demonstrate further control effectiveness of periodic SP via SDP. Controllable VE core modulus of SP is distributed periodically in space. Differential equations for coupling vibration of periodic SP with supported mass are derived and transformed into multi-dimensional system equations with parameters as nonlinear functions of bounded control. The OBPC problem is established by the system equations and performance index with bound constraint. Then, an SDP equation is derived according to the SDP principle. The OBPC law is obtained from the SDP equation under bound constraint. Optimally controlled responses are calculated and compared with passively controlled responses to evaluate control effectiveness. Numerical results on responses and statistics of SP via the proposed OBPC show further remarkable control effectiveness.

The fashion apparel industry is facing an increasingly growing demand, compounded by the short sales lifecycle and strong seasonality of clothing, posing significant challenges to inventory management in the retail sector. Despite some retailers like Uniqlo and Zara implementing inventory management and dynamic pricing strategies, challenges persist due to the dynamic nature of fashion trends and the stochastic factors affecting inventory. To address these issues, we construct a mathematical model based on the mathematical expression of the deterministic fashion level function, where the geometric Brownian motion, widely applied in finance, is initially utilized in the stochastic fashion level function. Drawing on research findings from deteriorating inventory management and stochastic optimization, we investigate the fluctuation of inventory levels, optimal dynamic pricing, optimal production rates, and profits—four crucial indicators—via Pontryagin’s maximum principle. Analytical solutions are derived, and the numerical simulation is provided to verify and compare the proposed model with deterministic fashion level function models. The model emphasizes the importance of considering stochastic factors in decision-making processes and provides insights to enhance profitability, inventory management, and sustainable consumption in the fashion product industry.

We present a two-country model (North and South) that describes the phenomenon of offshoring and reshoring. The model is a continuous time-controlled Markov chain with binary states. The main trade-off involves production costs and transaction costs between one country and another. In the first part of this paper, we identify the key parameters of the model: the difference in unit production costs between the two countries considered, the marginal cost of transitioning between countries, and the incentive paid by the North country to all companies that have not relocated at the end of the planning interval. The final goal of our paper is to understand how national tax incentives can influence this process.

Defined benefit (DB) pension plans are a primary type of pension schemes with the sponsor assuming most of the risks. Longevity-indexed bonds have been used to hedge or transfer risks in pension plans. Our objective is to study an aggregated DB pension plan’s optimal risk management problem focusing on minimizing the solvency risk over a finite time horizon and to investigate the investment strategies in a market, comprising a longevity-indexed bond and a risk-free asset, under stochastic nominal interest rates. Using the dynamic programming technique in the stochastic control problem, we obtain the closed-form optimal investment strategy by solving the corresponding Hamilton–Jacobi–Bellman (HJB) equation. In addition, a comparative analysis implicates that longevity-indexed bonds significantly reduce solvency risk compared to zero-coupon bonds, offering a strategic advantage in pension fund management. Besides the closed-form solution and the comparative study, another novelty of this study is the extension of actuarial liability (AL) and normal cost (NC) definitions, and we introduce the risk neutral valuation of liabilities in DB pension scheme with the consideration of mortality rate.

This paper presents a comprehensive framework addressing optimal nonlinear analysis and feedback control synthesis for nonlinear stochastic dynamical systems. The focus lies on establishing connections between stochastic Lyapunov theory and stochastic Hamilton–Jacobi–Bellman theory within a unified perspective. We demonstrate that the closed-loop nonlinear system’s asymptotic stability in probability is ensured through a Lyapunov function, identified as the solution to the steady-state form of the stochastic Hamilton–Jacobi–Bellman equation. This dual assurance guarantees both stochastic stability and optimality. Additionally, optimal feedback controllers for affine nonlinear systems are developed using an inverse optimality framework tailored to the stochastic stabilization problem. Furthermore, the paper derives stability margins for optimal and inverse optimal stochastic feedback regulators. Gain, sector, and disk margin guarantees are established for nonlinear stochastic dynamical systems controlled by nonlinear optimal and inverse optimal Hamilton–Jacobi–Bellman controllers.

We propose a novel alternative approach to our previous work (Ben Hammouda et al., 2023) to improve the efficiency of Monte Carlo (MC) estimators for rare event probabilities for stochastic reaction networks (SRNs). In the same spirit of Ben Hammouda et al. (2023), an efficient path-dependent measure change is derived based on a connection between determining optimal importance sampling (IS) parameters within a class of probability measures and a stochastic optimal control formulation, corresponding to solving a variance minimization problem. In this work, we propose a novel approach to address the encountered curse of dimensionality by mapping the problem to a significantly lower-dimensional space via a Markovian projection (MP) idea. The output of this model reduction technique is a low-dimensional SRN (potentially even one dimensional) that preserves the marginal distribution of the original high-dimensional SRN system. The dynamics of the projected process are obtained by solving a related optimization problem via a discrete L 2 regression. By solving the resulting projected Hamilton–Jacobi–Bellman (HJB) equations for the reduced-dimensional SRN, we obtain projected IS parameters, which are then mapped back to the original full-dimensional SRN system, resulting in an efficient IS-MC estimator for rare events probabilities of the full-dimensional SRN. Our analysis and numerical experiments reveal that the proposed MP-HJB-IS approach substantially reduces the MC estimator variance, resulting in a lower computational complexity in the rare event regime than standard MC estimators. [ABSTRACT FROM AUTHOR]

We present a method for the approximate propagation of mean and covariance of a probability distribution through ordinary differential equations (ODE) with discontinuous right-hand side. For piecewise affine systems, a normalization of the propagated probability distribution at every time step allows us to analytically compute the expectation integrals of the mean and covariance dynamics while explicitly taking into account the discontinuity. This leads to a natural smoothing of the discontinuity such that for relevant levels of uncertainty the resulting ODE can be integrated directly with standard schemes and it is neither necessary to prespecify the switching sequence nor to use a switch detection method. We then show how this result can be employed in the more general case of piecewise smooth functions based on a structure preserving linearization scheme. The resulting dynamics can be straightforwardly used within standard formulations of stochastic optimal control problems with chance constraints. [ABSTRACT FROM AUTHOR]

We propose an online algorithm for solving a class of continuous-state Markov decision processes. The algorithm combines classical Q-learning with an asynchronous averaging procedure, which allows Q-function estimates at sampled state–action pairs to be adaptively updated based on observations collected along a single sample trajectory. These estimates are then used to iteratively construct an interpolation-based function approximator of the Q-function. We prove the convergence of the algorithm and provide numerical results to illustrate its performance. • Proposed a model-free Q-learning algorithm for solving a class of infinite horizon Markov decision processes with continuous state spaces. • Proved the strong convergence of the algorithm when used in conjunction with a class of non-linear function approximators. • Illustrated the performance of the proposed algorithm through simulation comparison studies. [ABSTRACT FROM AUTHOR]