Search

For conflict-controlled dynamical systems satisfying the conditions of generalized uniqueness and uniform boundedness, the solvability of the minimax problem in the class of relaxed controls is studied. The issues of properness of such a relaxation are considered; i.e., the possibility of approximating relaxed controls in the space of strategic measures by embeddings of ordinary controls is analyzed. For this purpose, the dependence of the set of measures on the general marginal distribution specified on one of the factors of the base space is studied. The continuity of this dependence in the Hausdorff metric defined by the metric corresponding to the ∗ -weak topology in the space of measures is established. The density of embeddings of ordinary controls and control–disturbance pairs in sets of corresponding relaxed controls in the ∗ -weak topologies is also shown. [ABSTRACT FROM AUTHOR]

In deterministic and stochastic control theory, relaxed or randomized control policies allow for versatile mathematical analysis (on continuity, compactness, convexity, and approximations) to be applicable with no artificial restrictions on the classes of control policies considered, leading to very general existence results on optimal measurable policies under various setups and information structures. On relaxed controls, two studied topologies are the Young and Borkar (weak\ast) topologies on spaces of functions from a state/measurement space to the space of probability measures on control action spaces; the former via a weak convergence topology on probability measures on a product space with a fixed marginal on the input (state) space, and the latter via a weak\ast topology on randomized policies viewed as maps from states/measurements to the space of signed measures with bounded variation. We establish implication and equivalence conditions between the Young and Borkar topologies on control policies. We then show that, under some conditions, for a controlled Markov chain with standard Borel spaces the invariant measure is weakly continuous on the space of stationary control policies defined by either of these topologies. An implication is near-optimality of quantized stationary policies in state and actions or continuous stationary and deterministic policies for average cost control under two sets of continuity conditions (with either weak continuity in the state-action pair or strong continuity in the action for each state) on transition kernels. [ABSTRACT FROM AUTHOR]

We study a class of sampled stochastic optimization problems, where the underlying state process has diffusive dynamics of the mean-field type. We establish the existence of optimal relaxed controls when the sample set has finite size. The core of our paper is to prove, via Γ-convergence, that the minimizer of the finite sample relaxed problem converges to that of the limiting optimization problem as the sample size tends to infinity. We connect the limit of the sampled objective functional to the unique solution, in the trajectory sense, of a nonlinear Fokker-Planck-Kolmogorov equation in a random environment. We highlight the connection between the minimizers of our optimization problems and the optimal training weights of a deep residual neural network. [ABSTRACT FROM AUTHOR]

Nonconvex optimal control problems governed by evolution problems in infinite-dimensional spaces (as, e.g., parabolic boundary-value problems) needs a continuous (and possibly also smooth) extension on some (preferably convex) compactification, called relaxation, to guarantee existence of their solutions and to facilitate analysis by relatively conventional tools. When the control is valued in some subsets of Lebesgue spaces, the usual extensions are either too coarse (allowing in fact only very restricted nonlinearities) or too fine (being nonmetrizable). To overcome these drawbacks, a compromising convex compactification is here devised, combining classical techniques for Young measures with Choquet theory. This is applied to parabolic optimal control problems as far as existence and optimality conditions concerns. [ABSTRACT FROM AUTHOR]

Linear formulations for the occupation measures of a diffusion process with singular boundary behavior and an arbitrary but fixed control process are motivated and derived. Under specific assumptions on the control process, the existence and uniqueness of solutions to these linear formulations for both long-term average or discounted cost criteria in infinite time are shown. In particular, absolute continuity for the occupation measure of the nonsingular behavior is proven. Two examples of simple Brownian diffusion illustrate the applicability of the results on specific control problems, and their relevance for ongoing research in stochastic control is discussed. [ABSTRACT FROM AUTHOR]

We study a family of mean field games with a state variable evolving as a multivariate jump–diffusion process. The jump component is driven by a Poisson process with a time-dependent intensity function. All coefficients, i.e. drift, volatility and jump size, are controlled. Under fairly general conditions, we establish existence of a solution in a relaxed version of the mean field game and give conditions under which the optimal strategies are in fact Markovian, hence extending to a jump–diffusion setting previous results established in Lacker (2015). The proofs rely upon the notions of relaxed controls and martingale problems. Finally, to complement the abstract existence results, we study a simple illiquid inter-bank market model, where the banks can change their reserves only at the jump times of some exogenous Poisson processes with a common constant intensity, and provide some numerical results. [ABSTRACT FROM AUTHOR]

We study mean field games and corresponding N-player games in continuous time over a finite time horizon where the position of each agent belongs to a finite state space. As opposed to previous works on finite state mean field games, we use a probabilistic representation of the system dynamics in terms of stochastic differential equations driven by Poisson random measures. Under mild assumptions, we prove existence of solutions to the mean field game in relaxed open-loop as well as relaxed feedback controls. Relying on the probabilistic representation and a coupling argument, we show that mean field game solutions provide symmetric ε N -Nash equilibria for the N-player game, both in open-loop and in feedback strategies (not relaxed), with ε N ≤ constant N. Under stronger assumptions, we also find solutions of the mean field game in ordinary feedback controls and prove uniqueness either in case of a small time horizon or under monotonicity. [ABSTRACT FROM AUTHOR]

We consider an optimal boundary control problem described by a semilinear parabolic partial differential equation, with control and state constraints. Since this problem may have no classical solutions, it is reformulated in the relaxed form. The relaxed control problem is discretized by using a finite element method in space and a partially implicit scheme in time, while the controls are approximated by piecewise constant relaxed controls. We first state the necessary conditions for optimality for the continuous problem and the discrete relaxed problem. Next, under appropriate assumptions, we prove that accumulation points of sequences of optimal (resp. admissible and extremal) discrete relaxed controls are optimal (resp. admissible and extremal) for the continuous relaxed problem. [ABSTRACT FROM AUTHOR]

Existence theory of optimal relaxed control problem for a class of stochastic hereditary evolution equations driven by Lévy noise has been studied. We formulate the problem in the martingale sense of Stroock and Varadhan to establish existence of optimal controls. The construction of the solution is based on the classical Faedo–Galerkin approximation, the compactness method and the Jakubowski version of the Skorokhod theorem for nonmetric spaces, and certain compactness properties of the class of Young measures on Suslin metrizable control sets. As application of the abstract theory, Oldroyd and Jeffreys fluids have been studied and existence of optimal relaxed control is established. Existence and uniqueness of a strong solution and uniqueness in law for the two-dimensional Oldroyd and Jeffreys fluids are also shown. [ABSTRACT FROM AUTHOR]

A numerical method is proposed for a class of stochastic control problems including singular behavior. This method solves an infinite-dimensional linear program equivalent to the stochastic control problem using a finite element type approximation, which results in a solvable finite-dimensional program. The discretization scheme as well as the necessary assumptions are discussed, and a detailed convergence analysis for the discretization scheme is given. Its performance is illustrated by two examples featuring a long-term average cost criterion. [ABSTRACT FROM AUTHOR]

We consider an optimal control problem for systems defined by nonlinear Volterra integral equations, with state constraints. No convexity assumptions are made on the data, and the problem is transformed into its relaxed form. We prove the existence of an optimal relaxed control and derive necessary conditions for optimality in the form of a relaxed minimum principle of Pontryagin type. We then apply a mixed Frank-Wolfe penalty method which constructs sequences of relaxed controls converging to extremal controls for this problem. A numerical example is given. [ABSTRACT FROM AUTHOR]

In this work, we develop the necessary conditions of optimality for partially observed control problems governed by stochastic integrodifferential equations of neutral type on Hilbert spaces driven by relaxed controls. Under suitable conditions, we establish the existence and the uniqueness of mild solution which has a continuous modification. We prove the continuity of the control to the solution map. Then we consider an optimal control problem of Bolza type. Using these results we give necessary conditions of optimality for a class of stochastic integrodifferential equations of neutral type on Hilbert spaces. [ABSTRACT FROM AUTHOR]

This paper rigorously connects the problem of optimal control of McKean-Vlasov dynamics with large systems of interacting controlled state processes. Precisely, the empirical distributions of near-optimal control-state pairs for the n-state systems, as n tends to infinity, admit limit points in distribution (if the objective functions are suitably coercive), and every such limit is supported on the set of optimal control-state pairs for the McKean-Vlasov problem. Conversely, any distribution on the set of optimal control-state pairs for the McKean-Vlasov problem can be realized as a limit in this manner. Arguments are based on controlled martingale problems, which lend themselves naturally to existence proofs; along the way it is shown that a large class of McKean-Vlasov control problems admit optimal Markovian controls. [ABSTRACT FROM AUTHOR]

Neighborhoods of points in $mathbb{R}^n$ where a positive linear combination of $C^1$ vector fields sum to zero contain, generically, cyclic trajectories that switch between the vector fields. Such points are called stasis points, and the approximating switching cycle can be chosen so that the timing of the switches exactly matches the positive linear weighting. In the case of two vector fields, the stasis points form one-dimensional $C^1$ manifolds containing nearby families of two-cycles. The generic case of two flows in $mathbb{R}^3$ can be diffeomorphed to a standard form with cubic curves as trajectories.

In this paper we consider controlled McKean-Vlasov stochastic evolution equations on Hilbert spaces. We prove existence and uniqueness of solutions and regularity properties thereof. We use relaxed controls, adapted to a current of sub-sigma algebras generated by observable processes, and taking values from a Polish space. We introduce an appropriate topology based on weak star convergence. We prove continuous dependence of solutions on controls with respect to appropriate topologies. Theses results are then used to prove existence of optimal controls for Bolza problems. Then we develop the necessary conditions of optimality based on semi-martingale representation theory on Hilbert spaces. Next we show that the adjoint processes arising from the necessary conditions optimality can be constructed from the solution of certain BSDE. [ABSTRACT FROM AUTHOR]

Abstract: In this paper we develop the necessary conditions of optimality for partially observed control problems governed by stochastic neutral evolution equations on Hilbert spaces driven by relaxed controls. [Copyright &y& Elsevier]

In this paper we consider the question of optimal control for a class of stochastic evolution equations on infinite dimensional Hilbert spaces with controls appearing in both the drift and the diffusion operators. We con- sider relaxed controls (measure valued random processes) and briefly present some results on the question of existence of mild solutions including their regularity followed by a result on existence of partially observed optimal relaxed controls. Then we develop the necessary conditions of optimality for partially observed relaxed controls. This is the main topic of this paper. Further we present an algorithm for computation of optimal policies followed by a brief discussion on regular versus relaxed controls. The paper is con- cluded by an example of a non-convex problem which is readily solvable by our approach. [ABSTRACT FROM AUTHOR]

In this paper, we consider nonconvex control problems of stochastic differential equations driven by relaxed controls adapted, in the weak star sense, to a current of sigma algebras generated by observable processes. We cover in a unified way both continuous diffusion and jump processes. We present existence of optimal controls before we construct the necessary conditions of optimality (unlike some papers in this area) using only functional analysis. We develop a stochastic Hamiltonian system of equations on a rigorous basis using the semimartingale representation theory and the Riesz representation theorem, leading naturally to the existence of the adjoint process which satisfies a backward stochastic differential equation. In other words, our approach predicts the existence of the adjoint process as a natural consequence of Riesz representation theory ensuring at the same time the (weak star) measurability. This is unlike other papers, where the adjoint process is introduced before its existence is proved. We believe this is one of our major contributions in this paper. We also discuss the realizability of relaxed controls by regular controls using the Krein-Millman theorem. We believe this is another major contribution of this paper. We also believe that our approach is direct and easy to understand following simply the precise logic of functional analysis. [ABSTRACT FROM AUTHOR]

We consider a class of controlled queue length processes, in which the control allocates each server's effort among the several classes of customers requiring its service. Served customers are routed through the network according to (prescribed) routing probabilities. In the fluid rescaling, $X^{n}(t)=\frac{1}{n} X(nt)$, we consider the optimal control problem of minimizing the integral of an undiscounted positive running cost until the first time that X=0. Our main result uses weak convergence ideas to show that the optimal value functions V of the stochastic control problems for X( t) converge (as n→∞) to the optimal value V of a control problem for the limiting fluid process. This requires certain equicontinuity and boundedness hypotheses on { V}. We observe that these are essentially the same hypotheses that would be needed for the Barles-Perthame approach in terms of semicontinuous viscosity solutions. Sufficient conditions for these equicontinuity and boundedness properties are briefly discussed. [ABSTRACT FROM AUTHOR]

In this paper, the authors study an optimal control problem governed by a class of multistate ordinary differential equations in the absence of convexity. To overcome the difficulty that the corresponding approximate optimal control problem may have no solution, relaxed controls are introduced. With the help of relaxation theory, Pontryagin’s maximum principle for the optimal pairs of the original control problem is obtained. In the end of this paper, the authors discuss the application of the maximum principle by an example. [ABSTRACT FROM AUTHOR]

Neighborhoods of points in ℝn where a positive linear combination of C¹ vector fields sum to zero contain, generically, cyclic trajectories that switch between the vector fields. Such points are called stasis points, and the approximating switching cycle can be chosen so that the timing of the switches exactly matches the positive linear weighting. In the case of two vector fields, the stasis points form one-dimensional C¹ manifolds containing nearby families of two-cycles. The generic case of two flows in ℝ³ can be diffeomorphed to a standard form with cubic curves as trajectories. [ABSTRACT FROM AUTHOR]

In this paper, we present a result on relaxibility of partially observed control problems for infinite dimensional stochastic systems in a Hilbert space. This is motivated by the fact that measure valued controls, also known as relaxed controls, are difficult to construct practically and so one must inquire if it is possible to approximate the solutions corresponding to measure valued controls by those corresponding to ordinary controls. Our main result is the relaxation theorem which states that the set of solutions corresponding to ordinary controls is weakly dense in the set of solutions corresponding to relaxed controls. This is presented in Theorem 5.3 after giving some existence results on optimal controls for the infinite dimensional Zakai equation used for its proff. [ABSTRACT FROM AUTHOR]

In the present paper, we investigate an approximation technique for relaxed optimal control problems. We study control processes governed by ordinary differential equations in the presence of state, target, and integral constraints. A variety of approximation schemes have been recognized as powerful tools for the theoretical studying and practical solving of Infinite-dimensional optimization problems. On the other hand, theoretical approaches to the relaxed optimal control problem with constraints are not sufficiently advanced to yield numerically tractable schemes. The explicit approximation of the compact control set makes it possible to reduce the sophisticated relaxed problem to an auxiliary optimization problem. A given trajectory of the relaxed problem can be approximated by trajectories of the auxiliary problem. An optimal solution of the introduced optimization problem provides a basis for the construction of minimizing sequences for the original optimal control problem. We describe how to carry out the numerical calculations in the context of nonlinear programming and establish the convergence properties of the obtained approximations. [ABSTRACT FROM AUTHOR]

Abstract: We consider an optimal control problem described by nonlinear ordinary differential equations, with control and state constraints, including pointwise state constraints. Since this problem may have no classical solutions, it is also formulated in relaxed form. The classical control problem is then discretized by using the implicit midpoint scheme, while the controls are approximated by (not necessarily continuous) piecewise linear classical controls. We first study the behavior in the limit of properties of discrete optimality, and of discrete admissibility and extremality. We then apply a penalized gradient projection method to each discrete classical problem, and also a corresponding progressively refining combined discretization-optimization method to the continuous classical problem, thus reducing computing time and memory. We prove that accumulation points of sequences generated by these methods are admissible and extremal in some sense for the corresponding discrete or continuous, classical or relaxed, problem. For nonconvex problems whose solutions are nonclassical, we show that we can apply the above methods to the problem formulated in Gamkrelidze relaxed form. Finally, numerical examples are given. [Copyright &y& Elsevier]

We consider an optimal control problem described by nonlinear ordinary differential equations, with control and state constraints, including pointwise state constraints. Because no convexity assumptions are made, the problem may have no classical solutions, and it is reformulated in relaxed form. The relaxed control problem is then discretized by using the implicit midpoint scheme, while the controls are approximated by piecewise constant relaxed controls. We first study the behavior in the limit of properties of discrete relaxed optimality, and of discrete relaxed admissibility and extremality. We then apply a penalized conditional descent method to each discrete relaxed problem, and also a corresponding discrete method to the continuous relaxed problem that progressively refines the discretization during the iterations, thus reducing computing time and memory. We prove that accumulation points of sequences generated by these methods are admissible and extremal for the discrete or the continuous problem. Finally, numerical examples are given. [ABSTRACT FROM AUTHOR]

This work is devoted to near-optimal controls of large-scale discrete-time nonlinear dynamic systems driven by Markov chains; the underlying problem is to minimize an expected cost function. Our main goal is to reduce the complexity of the underlying systems. To achieve this goal, discrete-time control models under singularly-perturbed Markov chains are introduced. Using a relaxed control representation, our effort is devoted to finding near-optimal controls. Lumping the states in each irreducible class into a single state gives rise to a limit system. Applying near-optimal controls of the limit system to the original system, near-optimal controls of the original system are derived. [ABSTRACT FROM AUTHOR]

It is well known that singular problems may fail to have optimal solution in the class of “ordinary” (say, square-integrable) controls, even in the cases where the cost is bounded from below. In this paper, we suggest a method for overcoming this difficulty by defining an order r of singularity of the problem and extending both the input-trajectory map and the cost functional to an adequate subspace of the Sobolev space H_r-r. We show that the extended problem has a minimum if and only if the infimum of the original problem is finite. The extended problem can be transformed in a “natural” way into a regular L-Q problem with strictly smaller controllable space and (possibly) smaller control space. We use this transformation to describe the structure of relaxed optimal controls and the corresponding relaxed trajectories. We provide a method for computing optimal relaxed solutions from the solution of an adequate Riccati differential equation. We also show how the relaxed minimizers can be approximated by square-integrable functions. [ABSTRACT FROM AUTHOR]

The goal of this note is to correct the gap in the proof of Lemma 3.1, p. 3239, in the paper [N. U. Ahmed and C. D. Charalambous, SIAM J. Control Optim., 51 (2013), pp. 3235-3257]. The gap was pointed out by Dr. Qingxin Meng, Department of Mathematics, Huzhou University, Zhejiang, China [ABSTRACT FROM AUTHOR]