Search

In this paper, we obtain the maximum principle for optimal controls of stochastic systems with jumps by introducing a new method of variation. The control is allowed to enter both diffusion and jump term and the control domain need not to be convex.

In this paper, we establish spectral inequalities on measurable sets of positive Lebesgue measure for the Stokes operator, as well as an observability inequalities on space-time measurable sets of positive measure for non-stationary Stokes system. Furthermore, we provide their applications in the theory of shape optimization and time optimal control problems.

International audience; We are concerned with the design of Model Predictive Control (MPC) schemes such that asymptotic stability of the resulting closed loop is guaranteed even if the linearization at the desired set point fails to be stabilizable. Therefore, we propose to construct the stage cost based on the homogeneous approximation and rigorously show that applying MPC yields an asymp-totically stable closed-loop behavior if the homogeneous approximation is asymptotically null controllable. To this end, we verify cost controllability-a condition relating the current state, the stage cost, and the growth of the value function w.r.t. time-for this class of systems in order to provide stability and performance guarantees for the proposed MPC scheme without stabilizing terminal costs or constraints.

We consider average-cost Markov decision processes (MDPs) with Borel state spaces, countable, discrete action spaces, and strictly unbounded one-stage costs. For the minimum pair approach, we introduce a new majorization condition on the state transition stochastic kernel, in place of the commonly required continuity conditions on the MDP model. We combine this majorization condition with Lusin's theorem to prove the existence of a stationary minimum pair, i.e., a stationary policy paired with an invariant probability measure induced on the state space, with the property that the pair attains the minimum long-run average cost over all policies and initial distributions. We also establish other optimality properties of a stationary minimum pair, and for the stationary policy in such a pair, under additional recurrence or regularity conditions, we prove its pathwise optimality and strong optimality. Our results can be applied to a class of countable action space MDPs in which the dynamics and one-stage costs are discontinuous with respect to the state variable., 25 pages; to appear in SIAM Journal on Control and Optimization. This paper extends the author's earlier results in Section 4 of arXiv:1901.03374v1

The paper investigates sufficient conditions on a differential inclusion which guarantee that the origin is a finite time stable equilibrium, namely a weak local one, a weak global one or a strong local one. The analysis relies on the existence of a Lyapunov function. A new Gronwall type results are used to estimate the settling time. An example of a neural network which is finite-time stable is given

For sequential stochastic control problems with standard Borel measurement and control action spaces, we introduce a general (universally applicable) dynamic programming formulation, establish its well-posedness, and provide new existence results for optimal policies. Our dynamic program builds in part on Witsenhausen's standard form, but with a different formulation for the state, action, and transition dynamics. Using recent results on measurability properties of strategic measures in decentralized control, we obtain a standard Borel controlled Markov model. This allows for a well-defined dynamic programming recursion through universal measurability properties of the value functions for each time stage. In addition, new existence results are obtained for optimal policies in decentralized stochastic control. These state that for a static team with independent measurements, it suffices for the cost function to be continuous (only) in the actions for the existence of an optimal policy under mild compactness (or tightness) conditions. These also apply to dynamic teams which admit static reductions with independent measurements through a change of measure transformation. We show through a counterexample that weaker conditions may not lead to existence of an optimal team policy. The paper's existence results generalize those previously reported in the literature. A summary of and comparison with previously reported results and some applications are presented., To appear in SIAM Journal on Control and Optimization

We address the variational problem for the generalized principal eigenvalue on $\mathbb{R}^d$ of linear and semilinear elliptic operators associated with nondegenerate diffusions controlled through the drift. We establish the Collatz-Wielandt formula for potentials that vanish at infinity under minimal hypotheses, and also for general potentials under blanket geometric ergodicity assumptions. We also present associated results having the flavor of a refined maximum principle., Comment: 19 pages

In this paper we study a bilinear optimal control problem associated to a 3D chemo-repulsion model with linear production. We prove the existence of weak solutions and we establish a regularity criterion to get global in time strong solutions. As a consequence, we deduce the existence of a global optimal solution with bilinear control and, using a Lagrange multipliers theorem, we derive first-order optimality conditions for local optimal solutions., 41 pages. arXiv admin note: text overlap with arXiv:1806.10076

In this paper a priori error estimates are derived for full discretization (in space and time) of time-optimal control problems. Various convergence results for the optimal time and the control variable are proved under different assumptions. Especially the case of bang-bang controls is investigated. Numerical examples are provided to illustrate the results.

We construct an approximate public-signal correlated equilibrium for a nonzero-sum differential game in the class of stochastic strategies with memory. The construction is based on a solution of an auxiliary nonzero-sum continuous-time stochastic game. This class of games includes stochastic differential games and continuous-time Markov games. Moreover, we study the limit of approximate equilibrium outcomes in the case when the auxiliary stochastic games tend to the original deterministic one. We show that it lies in the convex hull of the set of equilibrium values provided by deterministic punishment strategies., 35 pages

This paper is concerned with the generation of Gaussian invariant states in cascades of open quantum harmonic oscillators governed by linear quantum stochastic differential equations. We carry out infinitesimal perturbation analysis of the covariance matrix for the invariant Gaussian state of such a system and the related purity functional subject to inaccuracies in the energy and coupling matrices of the subsystems. This leads to the problem of balancing the state-space realizations of the component oscillators through symplectic similarity transformations in order to minimize the mean square sensitivity of the purity functional to small random perturbations of the parameters. This results in a quadratic optimization problem with an effective solution in the case of cascaded one-mode oscillators, which is demonstrated by a numerical example. We also discuss a connection of the sensitivity index with classical statistical distances and outline infinitesimal perturbation analysis for translation invariant cascades of identical oscillators. The findings of the paper are applicable to robust state generation in quantum stochastic networks., Comment: 41 pages, 3 figures, 1 table

This paper studies Markov decision processes (MDPs) with atomless initial state distributions and atomless transition probabilities. Such MDPs are called atomless. The initial state distribution is considered to be fixed. We show that for discounted MDPs with bounded one-step reward vector-functions, for each policy there exists a deterministic (that is, nonrandomized and stationary) policy with the same performance vector. This fact is proved in the paper for a more general class of uniformly absorbing MDPs with expected total rewards, and then it is extended under certain assumptions to MDPs with unbounded rewards. For problems with multiple criteria and constraints, the results of this paper imply that for atomless MDPs studied in this paper it is sufficient to consider only deterministic policies, while without the atomless assumption it is well-known that randomized policies can outperform deterministic ones. We also provide an example of an MDP demonstrating that if a vector measure is defined on a standard Borel space, then Lyapunov's convexity theorem is a special case of the described results.

We study an infinite-horizon discrete-time optimal stopping problem under non-exponential discounting. A new method, which we call the iterative approach, is developed to find subgame perfect Nash equilibria. When the discount function induces decreasing impatience, we establish the existence of an equilibrium through fixed-point iterations. Moreover, we show that there exists a unique optimal equilibrium, which generates larger value than any other equilibrium does at all times. To the best of our knowledge, this is the first time a dominating subgame perfect Nash equilibrium is shown to exist in the literature of time-inconsistency.

We study Mean Field stochastic control problems where the cost function and the state dynamics depend upon the joint distribution of the controlled state and the control process. We prove suitable versions of the Pontryagin stochastic maximum principle, both in necessary and in sufficient form, which extend the known conditions to this general framework. Furthermore, we suggest a variational approach to study a weak formulation of these control problems. We show a natural connection between this weak formulation and optimal transport on path space, which inspires a novel discretization scheme., Comment: We changed the title, added an example, and suggest a discretization scheme

In a framework close to the one developed by Holmstr\"om and Milgrom [44], we study the optimal contracting scheme between a Principal and several Agents. Each hired Agent is in charge of one project, and can make efforts towards managing his own project, as well as impact (positively or negatively) the projects of the other Agents. Considering economic Agents in competition with relative performance concerns, we derive the optimal contracts in both first best and moral hazard settings. The enhanced resolution methodology relies heavily on the connection between Nash equilibria and multidimensional quadratic BSDEs. The optimal contracts are linear and each agent is paid a fixed proportion of the terminal value of all the projects of the firm. Besides, each Agent receives his reservation utility, and those with high competitive appetence are assigned less volatile projects, and shall even receive help from the other Agents. From the principal point of view, it is in the firm interest in our model to strongly diversify the competitive appetence of the Agents., Comment: 36 pages

In this paper, we design Dirichlet-Neumann boundary feedback controllers for the Korteweg-de Vries (KdV) equation that act at the right endpoint of the domain. The length of the domain is allowed to be critical. Constructing backstepping controllers that act at the right endpoint of the domain is more challenging than its left endpoint counterpart. The standard application of the backstepping method fails, because corresponding kernel models become overdetermined. In order to deal with this difficulty, we introduce the pseudo-backstepping method, which uses a pseudo-kernel that satisfies all but one desirable boundary condition. Moreover, various norms of the pseudo-kernel can be controlled through a parameter in one of its boundary conditions. We prove that the boundary controllers constructed via this pseudo-kernel still exponentially stabilize the system with the cost of a low exponential rate of decay. We show that a single Dirichlet controller is sufficient for exponential stabilization with a slower rate of decay. We also consider a second order feedback law acting at the right Dirichlet boundary condition. We show that this approach works if the main equation includes only the third order term, while the same problem remains open if the main equation involves the first order and/or the nonlinear term(s). At the end of the paper, we give numerical simulations to illustrate the main result., Comment: 23 pages, 11 figures, 1 table. In this version, an additional reference has been added and discussed within the text. Also some typos were corrected

Synchronization in networks of coupled oscillators is a widely studied topic with extensive scientific and engineering applications. In this paper, we study the frequency synchronization problem for networks of Kuramoto oscillators with arbitrary topology and heterogeneous edge weights. We propose a novel equivalent transcription for the equilibrium synchronization equation. Using this transcription, we develop a power series expansion to compute the synchronized solution of the Kuramoto model as well as a sufficient condition for the strong convergence of this series expansion. Truncating the power series provides (i) an efficient approximation scheme for computing the synchronized solution, and (ii) a simple-to-check, statistically-correct hierarchy of increasingly accurate synchronization tests. This hierarchy of tests provides a theoretical foundation for and generalizes the best-known approximate synchronization test in the literature. Our numerical experiments illustrate the accuracy and the computational efficiency of the truncated series approximation compared to existing iterative methods and existing synchronization tests.

International audience; Motivated by the result of invariance of regular-boundary open sets in \cite{CannarsaDaPratoFrankowska2009} and multi-stability issues in gene networks, our paper focuses on three closely related aims. First, we give a necessary local Lipschitz-like condition in order to expect invariance of open sets (for deterministic systems). Comments on optimality are provided via examples. Second, we provide a border avoidance (near-viability) counterpart of \cite{CannarsaDaPratoFrankowska2009} for controlled Brownian diffusions and piecewise deterministic switched Markov processes (PDsMP). We equally discuss to which extent Lipschitz-continuity of the driving coefficients is needed. Finally, by applying the theoretical result on PDsMP to Hasty's model of bacteriophage (\cite{hasty_pradines_dolnik_collins_00}, \cite{crudu_debussche_radulescu_09}), we show the necessity of explicit modeling for the environmental cue triggering lysis.

S. Hansen and E. Zuazua [SIAM J. Cont. Optim., 1995] studied the problem of exact controllability of two strings connected by a point mass with constant physical coefficients. In this paper we study the same problem with variable physical coefficients. This system is generated by the following equations $$\rho(x) u_{tt}=(\sigma(x) u_{x})_{x}-q(x)u,~~~~x\in (-1,0)\cup (0,1),~t>0,$$ $$Mu_{tt}(0,t)+\sigma_{1}(0)u_{x}(0^{-},t)-\sigma_{2}(0)u_{x}(0^{+},t)=0,~~~t>0,$$ with Dirichlet boundary condition on the left end and a control acts on the right end. We prove that this system is exactly controllable in an asymmetric space for the control time $T> 2\int_{-1}^{1}(\frac{\rho(x)}{\sigma(x)})^{\frac{1}{2}}dx$. We establish the equivalence between a suitable asymmetric norm of the initial data and the $L^{2}(0,T)$-norm of $u_{x}(1,t)$. Our approach is mainly based on a detailed spectral analysis and the theory of divided differences. More precisely, we prove that the spectral gap tends to zero with a precise asymptotic estimate., Comment: 34pages and 0 figures

We consider a controlled evolution problem for a set $\Omega(t)\in\mathbb{R}^d$, originally motivated by a model where a dog controls a flock of sheep. Necessary conditions and sufficient conditions are given, in order that the evolution be completely controllable. Similar techniques are then applied to the approximation of a sweeping process. Under suitable assumptions, we prove that there exists a control function such that the corresponding evolution of the set $\Omega(t)$ is arbitrarily close to the one determined by the sweeping process., Comment: 28 pages, 3 figures

We propose an approximation scheme for a class of semilinear parabolic equations that are convex and coercive in their gradients. Such equations arise often in pricing and portfolio management in incomplete markets and, more broadly, are directly connected to the representation of solutions to backward stochastic differential equations. The proposed scheme is based on splitting the equation in two parts, the first corresponding to a linear parabolic equation and the second to a Hamilton-Jacobi equation. The solutions of these two equations are approximated using, respectively, the Feynman-Kac and the Hopf-Lax formulae. We establish the convergence of the scheme and determine the convergence rate, combining Krylov's shaking coefficients technique and Barles-Jakobsen's optimal switching approximation., Comment: 24 pages

The convergence properties of adaptive systems in terms of excitation conditions on the regressor vector are well known. With persistent excitation of the regressor vector in model reference adaptive control the state error and the adaptation error are globally exponentially stable, or equivalently, exponentially stable in the large. When the excitation condition however is imposed on the reference input or the reference model state it is often incorrectly concluded that the persistent excitation in those signals also implies exponential stability in the large. The definition of persistent excitation is revisited so as to address some possible confusion in the adaptive control literature. It is then shown that persistent excitation of the reference model only implies local persistent excitation (weak persistent excitation). Weak persistent excitation of the regressor is still sufficient for uniform asymptotic stability in the large, but not exponential stability in the large. We show that there exists an infinite region in the state-space of adaptive systems where the state rate is bounded. This infinite region with finite rate of convergence is shown to exist not only in classic open-loop reference model adaptive systems, but also in a new class of closed-loop reference model adaptive systems., Comment: 22 pages, 5 figures

We consider an infinite horizon optimal control problem for a continuous-time Markov chain $X$ in a finite set $I$ with noise-free partial observation. The observation process is defined as $Y_t = h(X_t)$, $t \geq 0$, where $h$ is a given map defined on $I$. The observation is noise-free in the sense that the only source of randomness is the process $X$ itself. The aim is to minimize a discounted cost functional and study the associated value function $V$. After transforming the control problem with partial observation into one with complete observation (the separated problem) using filtering equations, we provide a link between the value function $v$ associated with the latter control problem and the original value function $V$. Then, we present two different characterizations of $v$ (and indirectly of $V$): on one hand as the unique fixed point of a suitably defined contraction mapping and on the other hand as the unique constrained viscosity solution (in the sense of Soner) of a HJB integro-differential equation. Under suitable assumptions, we finally prove the existence of an optimal control

We consider a class of linear-quadratic-Gaussian mean-field games with a major agent and considerable heterogeneous minor agents in the presence of mean-field interactions. The individual admissible controls are constrained in closed convex subsets $\Gamma_{k}$ of $\mathbb{R}^{m}.$ The decentralized strategies for individual agents and consistency condition system are represented in an unified manner through a class of mean-field forward-backward stochastic differential equations involving projection operators on $\Gamma_{k}$. The well-posedness of consistency system is established in both the local and global cases by the contraction mapping and discounting method respectively. Related $\varepsilon-$Nash equilibrium property is also verified., Comment: 40 pages

With the development of robotics, there are growing needs for real time motion planning. However, due to obstacles in the environment, the planning problem is highly non-convex, which makes it difficult to achieve real time computation using existing non-convex optimization algorithms. This paper introduces the convex feasible set algorithm (CFS) which is a fast algorithm for non-convex optimization problems that have convex costs and non-convex constraints. The idea is to find a convex feasible set for the original problem and iteratively solve a sequence of subproblems using the convex constraints. The feasibility and the convergence of the proposed algorithm are proved in the paper. The application of this method on motion planning for mobile robots is discussed. The simulations demonstrate the effectiveness of the proposed algorithm., in SIAM Journal on Control and Optimization

This article studies a portfolio optimization problem, where the market consisting of several stocks is modeled by a multi-dimensional jump-diffusion process with age-dependent semi-Markov modulated coefficients. We study risk sensitive portfolio optimization on the finite time horizon. We study the problem by using a probabilistic approach to establish the existence and uniqueness of the classical solution to the corresponding Hamilton-Jacobi-Bellman (HJB) equation. We also implement a numerical scheme to investigate the behavior of solutions for different values of the initial portfolio wealth, the maturity, and the risk of aversion parameter., 29 pages, 3 figures

This paper presents the equivalence between minimal time and minimal norm control problems for internally controlled heat equations. The target is an arbitrarily fixed bounded, closed and convex set with a nonempty interior in the state space. This study differs from [G. Wang and E. Zuazua, \textit{On the equivalence of minimal time and minimal norm controls for internally controlled heat equations}, SIAM J. Control Optim., 50 (2012), pp. 2938-2958] where the target set is the origin in the state space. When the target set is the origin or a ball, centered at the origin, the minimal norm and the minimal time functions are continuous and strictly decreasing, and they are inverses of each other. However, when the target is located in other place of the state space, the minimal norm function may be no longer monotonous and the range of the minimal time function may not be connected. These cause the main difficulty in our study. We overcome this difficulty by borrowing some idea from the classical raising sun lemma (see, for instance, Lemma 3.5 and Figure 5 on Pages 121-122 in [E. M. Stein and R. Shakarchi, \textit{Real Analysis: Measure Theory, Integration, and Hilbert Spaces}, Princeton University Press, 2005]).

This paper deals with a general form of variational problems in Banach spaces which encompasses variational inequalities as well as minimization problems. We prove a characterization of local error bounds for the distance to the (primal-dual) solution set and give a sufficient condition for such an error bound to hold. In the second part of the paper, we consider an algorithm of augmented Lagrangian type for the solution of such variational problems. We give some global convergence properties of the method and then use the error bound theory to provide estimates for the rate of convergence and to deduce boundedness of the sequence of penalty parameters. Finally, numerical results for optimal control, Nash equilibrium problems, and elliptic parameter estimation problems are presented., Comment: 27 pages

In this work, we design distributed control laws for spatial self-organization of multi-agent swarms in 1D and 2D spatial domains. The objective is to achieve a desired density distribution over a simply-connected spatial domain. Since individual agents in a swarm are not themselves of interest and we are concerned only with the macroscopic objective, we view the network of agents in the swarm as a discrete approximation of a continuous medium and design control laws to shape the density distribution of the continuous medium. The key feature of this work is that the agents in the swarm do not have access to position information. Each individual agent is capable of measuring the current local density of agents and can communicate with its spatial neighbors. The network of agents implement a Laplacian-based distributed algorithm, which we call pseudo-localization, to localize themselves in a new coordinate frame, and a distributed control law to converge to the desired spatial density distribution. We start by studying self-organization in one-dimension, which is then followed by the two-dimensional case.

In this paper we consider a control problem for a Partially Observable Piecewise Deterministic Markov Process of the following type: After the jump of the process the controller receives a noisy signal about the state and the aim is to control the process continuously in time in such a way that the expected discounted cost of the system is minimized. We solve this optimization problem by reducing it to a discrete-time Markov Decision Process. This includes the derivation of a filter for the unobservable state. Imposing sufficient continuity and compactness assumptions we are able to prove the existence of optimal policies and show that the value function satisfies a fixed point equation. A generic application is given to illustrate the results.

In this paper we study the fully nonlinear stochastic Hamilton-Jacobi-Bellman (HJB) equation for the optimal stochastic control problem of stochastic differential equations with random coefficients. The notion of viscosity solution is introduced, and we prove that the value function of the optimal stochastic control problem is the maximal viscosity solution of the associated stochastic HJB equation. For the superparabolic cases when the diffusion coefficients are deterministic functions of time, states and controls, the uniqueness is addressed as well., Comment: 24 pages

Hybrid systems with memory refer to dynamical systems exhibiting both hybrid and delay phenomena. While systems of this type are frequently encountered in many physical and engineering systems, particularly in control applications, various issues centered around the robustness of hybrid delay systems have not been adequately dealt with. In this paper, we establish some basic results on a framework that allows to study hybrid systems with memory through generalized concepts of solutions. In particular, we develop the basic existence of generalized solutions using regularity conditions on the hybrid data, which are formulated in a phase space of hybrid trajectories equipped with the graphical convergence topology. In contrast with the uniform convergence topology that has been often used, adopting the graphical convergence topology allows us to establish well-posedness of hybrid systems with memory. We then show that, as a consequence of well-posedness, pre-asymptotic stability of well-posed hybrid systems with memory is robust.

For a zero-sum stochastic game which does not satisfy the Isaacs condition, we provide a value function representation for an Isaacs-type equation whose Hamiltonian lies in between the lower and upper Hamiltonians, as a convex combination of the two. For the general case (i.e. the convex combination is time and state dependent) our representation amounts to a random change of the rules of the game, to allow each player at any moment to see the other player's action or not, according to a coin toss with probabilities of heads and tails given by the convex combination appearing in the PDE. If the combination is state independent, then the rules can be set all in advance, in a deterministic way. This means that tossing the coin along the game, or tossing it repeatedly right at the beginning leads to the same value. The representations are asymptotic, over time discretizations. Space discretization is possible as well, leading to similar results., Comment: Preliminary version

We study a control problem governed by a semilinear parabolic equation. The control is a measure that acts as the kernel of a possibly nonlocal time delay term and the functional includes a non-differentiable term with the measure-norm of the control. Existence, uniqueness and regularity of the solution of the state equation, as well as differentiability properties of the control-to-state operator are obtained. Next, we provide first order optimality conditions for local solutions. Finally, the control space is suitably discretized and we prove convergence of the solutions of the discrete problems to the solutions of the original problem. Several numerical examples are included to illustrate the theoretical results., Comment: 27 pages, 3 figures

This paper addresses an open problem in the area of linear quadratic optimal control. We consider the regular, infinite-horizon, stability-modulo-a-subspace, indefinite linear quadratic problem under the assumption that the dynamics are stabilizable. Our result generalizes previous works dealing with the same problem in the case of controllable dynamics. We explicitly characterize the unique solution of the algebraic Riccati equation that gives the optimal cost and optimal feedback control, as well as necessary and sufficient conditions for the existence of optimal controls., Comment: 16 pages

We prove that for linear, discrete, time-varying, deterministic system (perfect model) with noisy outputs, the Riccati transformation in the Kalman filter asymptotically bounds the rank of the forecast and the analysis error covariance matrices to be less than or equal to the number of non-negative Lyapunov exponents of the system. Further, the support of these error covariance matrices is shown to be confined to the space spanned by the unstable-neutral backward Lyapunov vectors, providing the theoretical justification for the methodology of the algorithms that perform assimilation only in the unstable-neutral subspace. The equivalent property of the autonomous system is investigated as a special case.

We consider the problem of finding model-independent bounds on the price of an Asian option, when the call prices at the maturity date of the option are known. Our methods differ from most approaches to model-independent pricing in that we consider the problem as a dynamic programming problem, where the controlled process is the conditional distribution of the asset at the maturity date. By formulating the problem in this manner, we are able to determine the model-independent price through a PDE formulation. Notably, this approach does not require specific constraints on the payoff function (e.g. convexity), and would appear to generalise to many related problems., Comment: Updated version with some technical changes, and a new appendix containing a proof of the DPP

We study the optimal control of general stochastic McKean-Vlasov equation. Such problem is motivated originally from the asymptotic formulation of cooperative equilibrium for a large population of particles (players) in mean-field interaction under common noise. Our first main result is to state a dynamic programming principle for the value function in the Wasserstein space of probability measures, which is proved from a flow property of the conditional law of the controlled state process. Next, by relying on the notion of differentiability with respect to probability measures due to P.L. Lions [32], and It{\^o}'s formula along a flow of conditional measures, we derive the dynamic programming Hamilton-Jacobi-Bellman equation, and prove the viscosity property together with a uniqueness result for the value function. Finally, we solve explicitly the linear-quadratic stochastic McKean-Vlasov control problem and give an application to an interbank systemic risk model with common noise., Comment: 33 pages, to appear in SIAM Journal on Control and Optimization

International audience; We prove that for a class of zero-sum differential games with incomplete information on both sides, the value admits a probabilistic representation as the value of a zero-sum stochastic differential game with complete information, where both players control a continuous martingale. A similar representation as a control problem over discontinuous martingales was known for games with incomplete information on one side (see Cardaliaguet-Rainer [8]), and our result is a continuous-time analog of the so called splitting-game introduced in Laraki [20] and Sorin [27] in order to analyze discrete-time models. It was proved by Cardaliaguet [4, 5] that the value of the games we consider is the unique solution of some Hamilton-Jacobi equation with convexity constraints. Our result provides therefore a new probabilistic representation for solutions of Hamilton-Jacobi equations with convexity constraints as values of stochastic differential games with unbounded control spaces and unbounded volatility.

We construct two error feedback controllers for robust output tracking and disturbance rejection of a regular linear system with nonsmooth reference and disturbance signals. We show that for sufficiently smooth signals the output converges to the reference at a rate that depends on the behaviour of the transfer function of the plant on the imaginary axis. In addition, we construct a controller that can be designed to achieve robustness with respect to a given class of uncertainties in the system, and present a novel controller structure for output tracking and disturbance rejection without the robustness requirement. We also generalize the internal model principle for regular linear systems with boundary disturbance and for controllers with unbounded input and output operators. The construction of controllers is illustrated with an example where we consider output tracking of a nonsmooth periodic reference signal for a two-dimensional heat equation with boundary control and observation, and with periodic disturbances on the boundary., Comment: 30 pages, 3 figures, to appear in SIAM Journal on Control & Optimization

A time-inconsistent stochastic optimal control problem with a recursive cost functional is studied. Equilibrium strategy is introduced, which is time-consistent and locally approximately optimal. By means of multiperson hierarchical differential games associated with partitions of the time interval, a family of approximate equilibrium strategy is constructed, and by sending the mesh size of the time interval partition to zero, an equilibrium Hamilton--Jacobi--Bellman (HJB) equation is derived through which the equilibrium value function can be identified and the equilibrium strategy can be obtained. Moreover, a well-posedness result of the equilibrium HJB equation is established under certain conditions, and a verification theorem is proved. Finally, an illustrative example is presented, and some comparisons of different definitions of equilibrium strategy are put in order.

This paper is the second part of our series of work to establish pointwise second-order necessary conditions for stochastic optimal controls. In this part, we consider the general cases, i.e., the control region is allowed to be nonconvex, and the control variable enters into both the drift and the diffusion terms of the control systems. By introducing four variational equations and four adjoint equations, we obtain the desired necessary conditions for stochastic singular optimal controls in the sense of Pontryagin-type maximum principle., Comment: 49 pages

We study an optimal execution problem in illiquid markets with both instantaneous and persistent price impact and stochastic resilience when only absolutely continuous trading strategies are admissible. In our model the value function can be described by a three-dimensional system of backward stochastic differential equations (BSDE) with a singular terminal condition in one component. We prove existence and uniqueness of a solution to the BSDE system and characterize both the value function and the optimal strategy in terms of the unique solution to the BSDE system. Our existence proof is based on an asymptotic expansion of the BSDE system at the terminal time that allows us to express the system in terms of a equivalent system with finite terminal value but singular driver.

The paper deals with a problem of control of a system characterized by the fact that the influence of controls on the dynamics of certain functions of state variables (called observables) is relatively weak and the rates of change of these observables are much slower than the rates of change of the state variables themselves. The contributions of the paper are twofold. Firstly, the averaged system whose solutions approximate the trajectories of the slow observables is introduced, and it is shown that the optimal value of the problem of optimal control with time discounting criterion considered on the solutions of the system with slow observables (this problem is referred to as perturbed) converges to the optimal value of the corresponding problem of optimal control of the averaged system. Secondly, a way how an asymptotically optimal control of the perturbed problem can be constructed on the basis of an optimal solution of the averaged problem is indicated, sufficient and necessary optimality conditions for the averaged problem are stated, and a way how a near optimal solution of the latter can be constructed numerically is outlined (the construction being illustrated with an example)., 40 pages, 4 figures

Motivated by a control problem of a certain queueing network we consider a control problem where the dynamics is constrained in the nonnegative orthant $\mathbb{R}_+$ of the $d$-dimensional Euclidean space and controlled by the reflections at the faces/boundaries. We define a discounted value function associated to this problem and show that the value function is a viscosity solution to a certain HJB equation in $\mathbb{R}_+$ with nonlinear Neumann type boundary condition. Under certain conditions, we also characterize this value function as the unique solution to this HJB equation., Comment: 32 pages, To appear in SICON

A new approach to solving two-point boundary value problems for a wave equation is developed. This new approach exploits the principle of stationary action to reformulate and solve such problems in the framework of optimal control. In particular, an infinite dimensional optimal control problem is posed so that the wave equation dynamics and temporal boundary data of interest are captured via the characteristics of the associated Hamiltonian and choice of terminal payoff respectively. In order to solve this optimal control problem for any such terminal payoff, and hence solve any two-point boundary value problem corresponding to the boundary data encapsulated by that terminal payoff, a fundamental solution to the optimal control problem is constructed. Specifically, the optimal control problem corresponding to any given terminal payoff can be solved via a max-plus convolution of this fundamental solution with the specified terminal payoff. Crucially, the fundamental solution is shown to be a quadratic functional that is defined with respect to the unique solution of a set of operator differential equations, and computable using spectral methods. An example is presented in which this fundamental solution is computed and applied to solve a two-point boundary value problem for the wave equation of interest., 30 pages, 5 figures, journal version submitted January 2015

For 1-D parabolic PDEs with disturbances at both boundaries and distributed disturbances we provide ISS estimates in various norms. Due to the lack of an ISS Lyapunov functional for boundary disturbances, the proof methodology uses (i) an eigenfunction expansion of the solution, and (ii) a finite-difference scheme. The ISS estimate for the sup-norm leads to a refinement of the well-known maximum principle for the heat equation. Finally, the obtained results are applied to quasi-static thermoelasticity models that involve nonlocal boundary conditions. Small-gain conditions that guarantee the global exponential stability of the zero solution for such models are derived., 32 pages, submitted to SIAM Journal on Control and Optimization for possible publication

For the minimization of a nonlinear cost functional $j$ under convex constraints the relaxed projected gradient process $\varphi_{k+1} = \varphi_{k} + \alpha_k(P_H(\varphi_{k}-\lambda_k \nabla_H j(\varphi_{k}))-\varphi_{k})$ is a well known method. The analysis is classically performed in a Hilbert space $H$. We generalize this method to functionals $j$ which are differentiable in a Banach space. Thus it is possible to perform e.g. an $L^2$ gradient method if $j$ is only differentiable in $L^\infty$. We show global convergence using Armijo backtracking in $\alpha_k$ and allow the inner product and the scaling $\lambda_k$ to change in every iteration. As application we present a structural topology optimization problem based on a phase field model, where the reduced cost functional $j$ is differentiable in $H^1\cap L^\infty$. The presented numerical results using the $H^1$ inner product and a pointwise chosen metric including second order information show the expected mesh independency in the iteration numbers. The latter yields an additional, drastic decrease in iteration numbers as well as in computation time. Moreover we present numerical results using a BFGS update of the $H^1$ inner product for further optimization problems based on phase field models.

We study a class of mean-field stochastic differential equations driven by a fractional Brownian motion with Hurst parameter $H\in(1/2,1)$ and a related stochastic control problem. We derive a Pontryagin type maximum principle and the associated adjoint mean-field backward stochastic differential equation driven by a classical Brownian motion, and we prove that under certain assumptions, which generalise the classical ones, the necessary condition for the optimality of an admissible control is also sufficient., Comment: 34 pages

We introduce a data-driven order reduction method for nonlinear control systems, drawing on recent progress in machine learning and statistical dimensionality reduction. The method rests on the assumption that the nonlinear system behaves linearly when lifted into a high (or infinite) dimensional feature space where balanced truncation may be carried out implicitly. This leads to a nonlinear reduction map which can be combined with a representation of the system belonging to a reproducing kernel Hilbert space to give a closed, reduced order dynamical system which captures the essential input-output characteristics of the original model. Empirical simulations illustrating the approach are also provided., Rewritten to improve readability. arXiv admin note: text overlap with arXiv:1011.2952