Search

Optimal control problem is typically solved by first finding the value function through Hamilton-Jacobi equation (HJE) and then taking the minimizer of the Hamiltonian to obtain the control. In this work, instead of focusing on the value function, we propose a new formulation for the gradient of the value function (value-gradient) as a decoupled system of partial differential equations in the context of continuous-time deterministic discounted optimal control problem. We develop an efficient iterative scheme for this system of equations in parallel by utilizing the properties that they share the same characteristic curves as the HJE for the value function. For the theoretical part, we prove that this iterative scheme converges linearly in $L_\alpha^2$ sense for some suitable exponent $\alpha$ in a weight function. For the numerical method, we combine characteristic line method with machine learning techniques. Specifically, we generate multiple characteristic curves at each policy iteration from an ensemble of initial states, and compute both the value function and its gradient simultaneously on each curve as the labelled data. Then supervised machine learning is applied to minimize the weighted squared loss for both the value function and its gradients. Experimental results demonstrate that this new method not only significantly increases the accuracy but also improves the efficiency and robustness of the numerical estimates, particularly with less amount of characteristics data or fewer training steps.

A risk management version of the classical investment-consumption problem known as Merton's problem in the finance literature is proposed. Risk is measured by variance, which introduces a nonlinear function of the expected value into the control problem. Standard stochastic theory cannot properly handle this type of nonlinear stochastic optimization problem. Therefore, we study this time-inconsistent problem within the mean field-type control framework. We derive the sufficient condition of optimality and solve the problem completely. Numerical results illustrating the effect of risk on optimal policies are also presented. Applications can be numerous, including all kinds of investment decisions or operations decisions.

We present a deep learning approximation, stochastic optimization based, method for wave kinetic equations. To build confidence in our approach, we apply the method to a Smoluchowski coagulation equation with multiplicative kernel for which an analytic solution exists. Our deep learning approach is then used to approximate the non-stationary solution to a 3-wave kinetic equation corresponding to acoustic wave systems. To validate the neural network approximation, we compare the decay rate of the total energy with previously obtained theoretical results. A finite volume solution is presented and compared with the present method.

The objective of this paper is two-fold. The first objective is to complete the former work of Bensoussan and Frehse [2] . One big limitation of this paper was the fact that they are systems of PDE. on a bounded domain. One can expect solutions to be bounded, since one looks for smooth solutions. This is a very important property for the development of the method. It is true also that solutions which exist in a bounded domain may fail to exist on R n , because of the lack of bounds. We give conditions so that the results of [2] can be extended to R n . The second objective is to consider the BSDE (Backward stochastic differential equations) version of the system of PDE. This is the objective of a more recent work of Xing and Zitkovic [8] . They consider systems of BSDE with quadratic growth, which is a well-known open problem in the BSDE literature. Since the BSDE are Markovian, the problem is equivalent to the analytic one. However, because of this motivation the analytic problem is in R n and not on a bounded domain. Xing and Zitkovic developed a probabilistic approach. The connection between the analytic problem and the BSDE is not apparent. Our objective is to show that the analytic approach can be completely translated into a probabilistic one. Nevertheless probabilistic concepts are also useful, after their conversion into the analytic framework. This is in particular true for the uniqueness result.

This paper considers a class of linear-quadratic-Gaussian (LQG) mean-field games (MFGs) with partial observation structure for individual agents. Unlike other literature, there are some special features in our formulation. First, the individual state is driven by some common-noise due to the external factor and the state-average thus becomes a random process instead of a deterministic quantity. Second, the sensor function of individual observation depends on state-average thus the agents are coupled in triple manner: not only in their states and cost functionals, but also through their observation mechanism. The decentralized strategies for individual agents are derived by the Kalman filtering and separation principle. The consistency condition is obtained which is equivalent to the wellposedness of some forward-backward stochastic differential equation (FBSDE) driven by common noise. Finally, the related \begin{document}$ \epsilon $\end{document} -Nash equilibrium property is verified.

In this paper, a novel method is developed for day-ahead daily river-flow forecasting based on convolutional neural network (CNN). The proposed method incorporates both spatial and temporal information to improve the forecasting performance. A CNN model is usually trained by minimizing the mean squared error which is, however, sensitive to few particularly large errors. This character of squared error loss function will result in a poor estimator. To tackle the problem, a robust loss function is proposed to train the CNN. To facilitate the training of CNNs for multiple sites forecasting, transfer learning is also applied in this study. With transfer learning, a new CNN inherits the structure and partial learnable parameters from a well-trained CNN to reduce the training complexity. The forecasting performance of the proposed method is validated with real data of four rivers by comparing with widely used benchmarking models including the autoregressive model, multilayer perception network, kernel ridge regression, radial basis function neural network, and generic CNN. Numerical results show that the proposed method performs best in terms of the root mean squared error, mean absolute error, and mean absolute percentage error. The two-sample Kolmogorov-Smirnov test is further applied to assess the confidence on the conclusion.

This paper develops a novel data-driven maximum power point tracking (MPPT) method, which is of two-fold, to benefit the power generation of photovoltaics (PV) systems facing variable partial shading conditions (PSCs). Under each PSC, the proposed MPPT utilizes a compact data-driven modeling process to develop the power-voltage ( P - V ) curve model via the natural cubic spline. Next, the proposed MPPT method develops a novel natural cubic spline guided iterative search process to update the P - V curve model having multiple peaks and to promptly obtain the global maximum power point (GMPP) under the considered PSC. This is a pioneer study which discusses a GMPPT algorithm using a natural cubic spline-based P - V curve model. The convergence of the MPP tracked by the proposed algorithm to the GMPP is theoretically ensured by the property of the natural cubic spline. The effectiveness and robustness of the proposed algorithm have been comprehensively evaluated via extensive simulation studies and experiments. Computational results demonstrate that the proposed algorithm is more efficient and effective to attain GMPPs under variable PSCs by comparing with recent MPPT methods using heuristic techniques, which are easily trapped into local MPP under variable PSCs.

In this study, we focus on the maintenance of an intrusion detection system (IDS) that attempts to discriminate between benign and malicious traffic arriving at a firm. An attack is more likely to successfully harm the firm if the ability of its IDS to discriminate between malicious and benign traffic is low, implying loopholes or vulnerabilities in the firm’s security. A novel aspect of this study is the modeling of both continuous degradation in system discrimination ability (drift) and the arrival of abrupt shocks that can suddenly lower discrimination ability. We model shocks to arrive randomly and cause a random decrease in discrimination ability. Furthermore, we prove the existence of a steady‐state level of discrimination ability that firms should strive to reach and maintain. When discrimination ability is below this steady‐state level, full effort must be exerted to reach it. We also compare our model with alternative settings, examine the impact of parameter estimation error, and study scenarios in which the arrival rate of malicious traffic is a function of the steady‐state discrimination ability chosen by the firm.

This paper studies an inventory management problem faced by an upstream supplier that is in a collaborative agreement, such as vendor-managed inventory (VMI), with a retailer. A VMI partnership provides the supplier an opportunity to manage inventory for the supply chain in exchange for point-of-sales (POS)- and inventory-level information from the retailer. However, retailers typically possess superior local market information and as has been the case in recent years, are able to capture and analyze customer purchasing behavior beyond the traditional POS data. Such analyses provide the retailer access to market signals that are otherwise hard to capture using POS information. We show and quantify the implication of the financial obligations of each party in VMI that renders communication of such important market signals as noncredible. To help institute a sound VMI collaboration, we propose learn and screen—a dynamic inventory mechanism—for the supplier to effectively manage inventory and information in the supply chain. The proposed mechanism combines the ability of the supplier to learn about market conditions from POS data (over multiple selling periods) and dynamically determine when to screen the retailer and acquire his private demand information. Inventory decisions in the proposed mechanism serve a strategic purpose in addition to their classic role of satisfying customer demand. We show that our proposed dynamic mechanism significantly improves the supplier’s expected profit and increases the efficiency of the overall supply chain operations under a VMI agreement. In addition, we determine the market conditions in which a strategic approach to VMI results in significant profit improvements for both firms, particularly when the retailer has high market power (i.e., when the supplier highly depends on the retailer) and when the supplier has relatively less knowledge about the end customer/market compared with the retailer. This paper was accepted by Gad Allon, operations management.

We consider a version of the stochastic inventory control problem for a spectrally positive Levy demand process, in which the inventory can only be replenished at independent exponential times. We ...

International audience; We model the expansion decision of a levered firm. Straight debt distorts both timing and scaling: the firm invests less and later than its all-equity financed counterpart. The inclusion of performance sensitivity in the debt contract mitigates such distortions. Moreover, performance sensitivity is consistent with firm value maximization within a standard trade-off theory of capital structure. As a result, our model rationalizes the widespread use of performance sensitive debt (PSD), especially amongst fast growth firms.

In this article, we study mean-field-type games with jump–diffusion and regime switching in which the payoffs and the state dynamics depend not only on the state–action profile of the decision-makers but also on a measure of the state–action pair. The state dynamics is a measure-dependent process with jump–diffusion and regime switching. We derive novel equilibrium systems to be solved. Two solution approaches are presented: (i) dynamic programming principle and (ii) stochastic maximum principle. Relationship between dual function and adjoint processes are provided. It is shown that the extension to the risk-sensitive case generates a nonlinearity to the adjoint process and it involves three other processes associated with the diffusion, jump and regime switching, respectively.

In this paper we characterize the feedback equilibrium of a general infinite-horizon Stackelberg--Nash differential game where the roles of the players are mixed. By mixed we mean that one player i...