Search

Following the risk-taking model of Seel and Strack, $n$ players decide when to stop privately observed Brownian motions with drift and absorption at zero. They are then ranked according to their level of stopping and paid a rank-dependent reward. We study the problem of a principal who aims to induce a desirable equilibrium performance of the players by choosing how much reward is attributed to each rank. Specifically, we determine optimal reward schemes for principals interested in the average performance and the performance at a given rank. While the former can be related to reward inequality in the Lorenz sense, the latter can have a surprising shape., To appear in SIAM Journal on Financial Mathematics

We consider the Connectivity Augmentation Problem (CAP), a classical problem in the area of Survivable Network Design. It is about increasing the edge-connectivity of a graph by one unit in the cheapest possible way. More precisely, given a $k$-edge-connected graph $G=(V,E)$ and a set of extra edges, the task is to find a minimum cardinality subset of extra edges whose addition to $G$ makes the graph $(k+1)$-edge-connected. If $k$ is odd, the problem is known to reduce to the Tree Augmentation Problem (TAP) -- i.e., $G$ is a spanning tree -- for which significant progress has been achieved recently, leading to approximation factors below $1.5$ (the currently best factor is $1.458$). However, advances on TAP did not carry over to CAP so far. Indeed, only very recently, Byrka, Grandoni, and Ameli (STOC 2020) managed to obtain the first approximation factor below $2$ for CAP by presenting a $1.91$-approximation algorithm based on a method that is disjoint from recent advances for TAP. We first bridge the gap between TAP and CAP, by presenting techniques that allow for leveraging insights and methods from TAP to approach CAP. We then introduce a new way to get approximation factors below $1.5$, based on a new analysis technique. Through these ingredients, we obtain a $1.393$-approximation algorithm for CAP, and therefore also TAP. This leads to the currently best approximation result for both problems in a unified way, by significantly improving on the above-mentioned $1.91$-approximation for CAP and also the previously best approximation factor of $1.458$ for TAP by Grandoni, Kalaitzis, and Zenklusen (STOC 2018). Additionally, a feature we inherit from recent TAP advances is that our approach can deal with the weighted setting when the ratio between the largest to smallest cost on extra links is bounded, in which case we obtain approximation factors below $1.5$.

This work is a follow-up and a complement to arXiv:1912.08899 [math.OC] for solving polynomial optimization problems (POPs). The chordal-TSSOS hierarchy that we propose is a new sparse moment-SOS framework based on term-sparsity and chordal extension. By exploiting term-sparsity of the input polynomials we obtain a two-level hierarchy of semidefinite programming relaxations. The novelty and distinguishing feature of such relaxations is to obtain quasi block-diagonal matrices obtained in an iterative procedure that performs chordal extension of certain adjacency graphs. The graphs are related to the terms arising in the original data and not to the links between variables. Various numerical examples demonstrate the efficiency and the scalability of this new hierarchy for both unconstrained and constrained POPs. The two hierarchies are complementary. While the former TSSOS arXiv:1912.08899 [math.OC] has a theoretical convergence guarantee, the chordal-TSSOS has superior performance but lacks this theoretical guarantee., 29 pages, 14 tables, 5 figures. arXiv admin note: text overlap with arXiv:1912.08899

In this paper, we develop convergence analysis of a modified line search method for objective functions whose value is computed with noise and whose gradient estimates are inexact and possibly random. The noise is assumed to be bounded in absolute value without any additional assumptions. We extend the framework based on stochastic methods from [Cartis and Scheinberg, 2018] which was developed to provide analysis of a standard line search method with exact function values and random gradients to the case of noisy functions. We introduce two alternative conditions on the gradient which when satisfied with some sufficiently large probability at each iteration, guarantees convergence properties of the line search method. We derive expected complexity bounds to reach a near optimal neighborhood for convex, strongly convex and nonconvex functions. The exact dependence of the convergence neighborhood on the noise is specified., Comment: 30 pages. arXiv admin note: text overlap with arXiv:1905.01332

This paper examines mean field linear-quadratic-Gaussian (LQG) social optimum control with volatility-uncertain common noise. The diffusion terms in the dynamics of agents contain an unknown volatility process driven by a common noise. We apply a robust optimization approach in which all agents view volatility uncertainty as an adversarial player. Based on the principle of person-by-person optimality and a two-step-duality technique for stochastic variational analysis, we construct an auxiliary optimal control problem for a representative agent. Through solving this problem combined with a consistent mean field approximation, we design a set of decentralized strategies, which are further shown to be asymptotically social optimal by perturbation analysis.

In this paper we give an algorithm to round the floating point output of a semidefinite programming solver to a solution over the rationals or a quadratic extension of the rationals. We apply this to get sharp bounds for packing problems, and we use these sharp bounds to prove that certain optimal packing configurations are unique up to rotations. In particular, we show that the configuration coming from the $\mathsf{E}_8$ root lattice is the unique optimal code with minimal angular distance $\pi/3$ on the hemisphere in $\mathbb R^8$, and we prove that the three-point bound for the $(3, 8, \vartheta)$-spherical code, where $\vartheta$ is such that $\cos \vartheta = (2\sqrt{2}-1)/7$, is sharp by rounding to $\mathbb Q[\sqrt{2}]$. We also use our machinery to compute sharp upper bounds on the number of spheres that can be packed into a larger sphere., Comment: 24 pages

We propose a novel learning-based framework for image reconstruction particularly designed for training without ground truth data, which has three major building blocks: energy-based learning, a patch-based Wasserstein loss functional, and shared prior learning. In energy-based learning, the parameters of an energy functional composed of a learned data fidelity term and a data-driven regularizer are computed in a mean-field optimal control problem. In the absence of ground truth data, we change the loss functional to a patch-based Wasserstein functional, in which local statistics of the output images are compared to uncorrupted reference patches. Finally, in shared prior learning, both aforementioned optimal control problems are optimized simultaneously with shared learned parameters of the regularizer to further enhance unsupervised image reconstruction. We derive several time discretization schemes of the gradient flow and verify their consistency in terms of Mosco convergence. In numerous numerical experiments, we demonstrate that the proposed method generates state-of-the-art results for various image reconstruction applications--even if no ground truth images are available for training., 37 pages, 19 figures

This paper presents an accelerated composite gradient (ACG) variant, referred to as the AC-ACG method, for solving nonconvex smooth composite minimization problems. As opposed to well-known ACG variants that are either based on a known Lipschitz gradient constant or a sequence of maximum observed curvatures, the current one is based on the average of all past observed curvatures. More specifically, AC-ACG uses a positive multiple of the average of all observed curvatures until the previous iteration as a way to estimate the "function curvature" at the current point and then two resolvent evaluations to compute the next iterate. In contrast to other variable Lipschitz estimation variants, e.g., the ones based on the maximum curvature, AC-ACG always accepts the aforementioned iterate regardless how poor the Lipschitz estimation turns out to be. Finally, computational results are presented to illustrate the efficiency of AC-ACG on both randomly generated and real-world problem instances., 28 pages

We introduce iterative methods named TriCG and TriMR for solving symmetric quasi-definite systems based on the orthogonal tridiagonalization process proposed by Saunders, Simon and Yip in 1988. TriCG and TriMR are tantamount to preconditioned Block-CG and Block-MINRES with two right-hand sides in which the two approximate solutions are summed at each iteration, but require less storage and work per iteration. We evaluate the performance of TriCG and TriMR on linear systems generated from the SuiteSparse Matrix Collection and from discretized and stablized Stokes equations. We compare TriCG and TriMR with SYMMLQ and MINRES, the recommended Krylov methods for symmetric and indefinite systems. In all our experiments, TriCG and TriMR terminate earlier than SYMMLQ and MINRES on a residual-based stopping condition with an improvement of up to 50% in terms of number of iterations. They also terminate more reliably than Block-CG and Block-MINRES. Experiments in quadruple and octuple precision suggest that loss of orthogonality in the basis vectors is significantly less pronounced in TriCG and TriMR than in Block-CG and Block-MINRES., Comment: 24 pages, 12 figures

We introduce BELLA, a locally superlinearly convergent Bregman forward-backward splitting method for minimizing the sum of two nonconvex functions, one of which satisfies a relative smoothness condition and the other one is possibly nonsmooth. A key tool of our methodology is the Bregman forward-backward envelope (BFBE), an exact and continuous penalty function with favorable first- and second-order properties, which enjoys a nonlinear error bound when the objective function satisfies a Lojasiewicz-type property. The proposed algorithm is of linesearch type over the BFBE along user-defined update directions and converges subsequentially to stationary points and globally under the Kurdyka-Lojasiewicz condition. Moreover, when the update directions are superlinear in the sense of Facchinei and Pang [Finite-Dimensional Variational Inequalities and Complementarity Problems, Volume I, Springer, New York, 2003], owing to the given nonlinear error bound unit stepsize is eventually always accepted and the algorithm attains superlinear convergence rates even when the limit point is a nonisolated minimum.

The paper studies the open-loop saddle point and the open-loop lower and upper values, as well as their relationship for two-person zero-sum stochastic linear-quadratic (LQ, for short) differential games with deterministic coefficients. It derives a necessary condition for the finiteness of the open-loop lower and upper values and a sufficient condition for the existence of an open-loop saddle point. It turns out that under the sufficient condition, a strongly regular solution to the associated Riccati equation uniquely exists, in terms of which a closed-loop representation is further established for the open-loop saddle point. Examples are presented to show that the finiteness of the open-loop lower and upper values does not ensure the existence of an open-loop saddle point in general. But for the classical deterministic LQ game, these two issues are equivalent and both imply the solvability of the Riccati equation, for which an explicit representation of the solution is obtained., Comment: 26 pages

We study a class of countably infinite linear programs (CILPs) whose feasible sets are bounded subsets of appropriately defined spaces of measures. The optimal value, optimal points, and minimal points of these CILPs can be approximated by solving finite-dimensional linear programs. We show how to construct finite-dimensional programs that lead to approximations with easy-to-evaluate error bounds, and we prove that the errors converge to zero as the size of the finite-dimensional programs approaches that of the original problem. We discuss the use of our methods in the computation of the stationary distributions, occupation measures, and exit distributions of Markov chains. Read More:https://epubs.siam.org/doi/10.1137/19M1268847 &nbsp

We introduce new parametrized classes of shape admissible domains in R^n , n $\ge$ 2, and prove that they are compact with respect to the convergence in the sense of characteristic functions, the Hausdorff sense, the sense of compacts and the weak convergence of their boundary volumes. The domains in these classes are bounded ($\epsilon$, $\infty$)-domains with possibly fractal boundaries that can have parts of any non-uniform Hausdorff dimension greater or equal to n -- 1 and less than n. We prove the existence of optimal shapes in such classes for maximum energy dissipation in the framework of linear acous-tics. A by-product of our proof is the result that the class of bounded ($\epsilon$, $\infty$)-domains with fixed $\epsilon$ is stable under Hausdorff convergence. An additional and related result is the Mosco convergence of Robin-type energy functionals on converging domains.

In statistical decision theory involving a single decision-maker, an information structure is said to be better than another one if for any cost function involving a hidden state variable and an action variable which is restricted to be conditionally independent from the state given some measurement, the solution value under the former is not worse than that under the latter. For finite spaces, a theorem due to Blackwell leads to a complete characterization on when one information structure is better than another. For stochastic games, in general, such an ordering is not possible since additional information can lead to equilibria perturbations with positive or negative values to a player. However, for zero-sum games in a finite probability space, P\k{e}ski introduced a complete characterization of ordering of information structures. In this paper, we obtain an infinite dimensional (standard Borel) generalization of P\k{e}ski's result. A corollary is that more information cannot hurt a decision maker taking part in a zero-sum game. We establish two supporting results which are essential and explicit though modest improvements on prior literature: (i) a partial converse to Blackwell's ordering in the standard Borel setup and (ii) an existence result for equilibria in zero-sum games with incomplete information., Comment: To appear in SIAM Journal on Control and Optimization. A short conference version appeared at the 2020 IEEE International Symposium on Information Theory

An optimal ergodic control problem (EC problem, for short) is investigated for a linear stochastic differential equation with quadratic cost functional. Constant nonhomogeneous terms, not all zero, appear in the state equation, which lead to the asymptotic limit of the state non-zero. Under the stabilizability condition, for any (admissible) closed-loop strategy, an invariant measure is proved to exist, which makes the ergodic cost functional well-defined and the EC problem well-formulated. Sufficient conditions, including those allowing the weighting matrices of cost functional to be indefinite, are introduced for finiteness and solvability for the EC problem. Some comparisons are made between the solvability of EC problem and the closed-loop solvability of stochastic linear quadratic optimal control problem in the infinite horizon. Regularized EC problem is introduced to be used to obtain the optimal value of the EC problem.

Problems involving control of large ensmebles of structurally identical dynamical systems, called \emph{ensemble control}, arise in numerous scientific areas from quantum control and robotics to brain medicine. In many of such applications, control can only be implemented at the population level, i.e., through broadcasting an input signal to all the systems in the population, and this new control paradigm challenges the classical systems theory. In recent years, considerable efforts have been made to investigate controllability properties of ensemble systems, and most works emphasized on linear and some forms of bilinear and nonlinear ensemble systems. In this paper, we study controllability of a broad class of bilinear ensemble systems defined on semisimple Lie groups, for which we define the notion of ensemble controllability through a Riemannian structure of the state space Lie group. Leveraging the Cartan decomposition of semisimple Lie algebras in representation theory, we develop a \emph{covering method} that decomposes the state space Lie group into a collection of Lie subgroups generating the Lie group, which enables the determination of ensemble controllability by controllability of the subsystems evolving on these Lie subgroups. Using the covering method, we show the equivalence between ensemble and classical controllability, i.e., controllability of each individual system in the ensemble implies ensemble controllability, for bilinear ensemble systems evolving on semisimple Lie groups. This equivalence makes the examination of controllability for infinite-dimensional ensemble systems as tractable as for a finite-dimensional single system., Comment: keywords: Ensemble control, Semisimple Lie groups, Approximation theory, Controllability, Infinite-dimensional Systems

We consider a distributionally robust Partially Observable Markov Decision Process (DR-POMDP), where the distribution of the transition-observation probabilities is unknown at the beginning of each decision period, but their realizations can be inferred using side information at the end of each period after an action being taken. We build an ambiguity set of the joint distribution using bounded moments via conic constraints and seek an optimal policy to maximize the worst-case (minimum) reward for any distribution in the set. We show that the value function of DR-POMDP is piecewise linear convex with respect to the belief state and propose a heuristic search value iteration method for obtaining lower and upper bounds of the value function. We conduct numerical studies and demonstrate the computational performance of our approach via testing instances of a dynamic epidemic control problem. Our results show that DR-POMDP can produce more robust policies under misspecified distributions of transition-observation probabilities as compared to POMDP, but has less costly solutions than robust POMDP. The DR-POMDP policies are also insensitive to varying parameter in the ambiguity set and to noise added to the true transition-observation probability values obtained at the end of each decision period.

Nonconvex optimal-control problems governed by evolution problems in infinite-dimensional spaces (as e.g. parabolic boundary-value problems) needs a continuous (and possibly also smooth) extension on some (preferably convex) compactification, called relaxation, to guarantee existence of their solutions and to facilitate analysis by relatively conventional tools. When the control is valued in some subsets of Lebesgue spaces, the usual extensions are either too coarse (allowing in fact only very restricted nonlinearities) or too fine (being nonmetrizable). To overcome these drawbacks, a compromising convex compactification is here devised, combining classical techniques for Young measures with Choquet theory. This is applied to parabolic optimal control problems as far as existence and optimality conditions concerns.

Recently, in a series of papers [32,38,39,41], the ratio of $\ell_1$ and $\ell_2$ norms was proposed as a sparsity inducing function for noiseless compressed sensing. In this paper, we further study properties of such model in the noiseless setting, and propose an algorithm for minimizing $\ell_1$/$\ell_2$ subject to noise in the measurements. Specifically, we show that the extended objective function (the sum of the objective and the indicator function of the constraint set) of the model in [32] satisfies the Kurdyka-Lojasiewicz (KL) property with exponent 1/2; this allows us to establish linear convergence of the algorithm proposed in [39, Eq. 11] under mild assumptions. We next extend the $\ell_1$/$\ell_2$ model to handle compressed sensing problems with noise. We establish the solution existence for some of these models under the spherical section property [37,44], and extend the algorithm in [39, Eq. 11] by incorporating moving-balls-approximation techniques [4] for solving these problems. We prove the subsequential convergence of our algorithm under mild conditions, and establish global convergence of the whole sequence generated by our algorithm by imposing additional KL and differentiability assumptions on a specially constructed potential function. Finally, we perform numerical experiments on robust compressed sensing and basis pursuit denoising with residual error measured by $ \ell_2 $ norm or Lorentzian norm via solving the corresponding $\ell_1$/$\ell_2$ models by our algorithm. Our numerical simulations show that our algorithm is able to recover the original sparse vectors with reasonable accuracy.

Controlling the shape and position of moving and pinned droplets on a solid surface is an important feature often found in microfluidic applications. In this work, we consider a well investigated phase field model including contact line dynamics as the state system for an (open-loop) optimal control problem. Here the spatially and temporally changeable contact angles between droplet and solid are considered as the control variables. We consider a suitable, energy stable, time discrete version of the state equation in our optimal control problem. We discuss regularity of the solution to the time discrete state equation and its continuity and differentiability properties. Furthermore, we show existence of solutions and state first order optimality conditions to the optimal control problem. We illustrate our results by actively pushing a droplet uphill against gravity in an optimal way.

Recently, there has been a surge of interest in designing variants of the classical Newton-CG in which the Hessian of a (strongly) convex function is replaced by suitable approximations. This is mainly motivated by large-scale finite-sum minimization problems that arise in many machine learning applications. Going beyond convexity, inexact Hessian information has also been recently considered in the context of algorithms such as trust-region or (adaptive) cubic regularization for general non-convex problems. Here, we do that for Newton-MR, which extends the application range of the classical Newton-CG beyond convexity to invex problems. Unlike the convergence analysis of Newton-CG, which relies on spectrum preserving Hessian approximations in the sense of L\"{o}wner partial order, our work here draws from matrix perturbation theory to estimate the distance between the subspaces underlying the exact and approximate Hessian matrices. Numerical experiments demonstrate a great degree of resilience to such Hessian approximations, amounting to a highly efficient algorithm in large-scale problems., Comment: 32 pages, 10 figures

A linear control system with quadratic cost functional over infinite time horizon is considered without assuming controllability/stabilizability condition and the global integrability condition for the nonhomogeneous term of the state equation and the weight functions in the linear terms in the running cost rate function. Classical approaches do not apply for such kind of problems. Existence and non-existence of overtaking optimal controls in various cases are established. Some concrete examples are presented. These results show that the overtaking optimality approach can be used to solve some of the above-mentioned problems and at the same time, the limitation of this approach is also revealed.

Motivated by the problem of bounding the number of iterations of the Simplex algorithm we investigate the possible lengths of monotone paths followed by the Simplex method inside the oriented graphs of polyhedra (oriented by the objective function). We consider both the shortest and the longest monotone paths and estimate the monotone diameter and height of polyhedra. Our analysis applies to transportation polytopes, matroid polytopes, matching polytopes, shortest-path polytopes, and the TSP, among others. We begin by showing that combinatorial cubes have monotone and Bland pivot height bounded by their dimension and that in fact all monotone paths of zonotopes are no larger than the number of edge directions of the zonotope. We later use this to show that several polytopes have polynomial-size pivot height, for all pivot rules. In contrast, we show that many well-known combinatorial polytopes have exponentially-long monotone paths. Surprisingly, for some famous pivot rules, e.g., greatest improvement and steepest edge, these same polytopes have polynomial-size simplex paths., Comment: 24 pages, 8 figures

We prove a small-gain theorem for interconnections of $n$ nonlinear heterogeneous input-to-state stable (ISS) control systems of a general nature, covering partial, delay and ordinary differential equations. Furthermore, for the same class of control systems, we derive small-gain theorems for asymptotic gain, uniform global stability and weak input-to-state stability properties. We show that our technique is applicable for different formulations of ISS property (summation, maximum, semimaximum) and discuss tightness of achieved small-gain theorems. Finally, we introduce variations of uniform asymptotic gain and uniform limit properties, which are particularly useful for small-gain arguments and characterize ISS in terms of these notions., Accepted to SICON, 2021

We propose an extension of a special form of gradient descent -- in the literature known as linearised Bregman iteration -- to a larger class of non-convex functions. We replace the classical (squared) two norm metric in the gradient descent setting with a generalised Bregman distance, based on a proper, convex and lower semi-continuous function. The algorithm's global convergence is proven for functions that satisfy the Kurdyka-\L ojasiewicz property. Examples illustrate that features of different scale are being introduced throughout the iteration, transitioning from coarse to fine. This coarse-to-fine approach with respect to scale allows to recover solutions of non-convex optimisation problems that are superior to those obtained with conventional gradient descent, or even projected and proximal gradient descent. The effectiveness of the linearised Bregman iteration in combination with early stopping is illustrated for the applications of parallel magnetic resonance imaging, blind deconvolution as well as image classification with neural networks., Comment: 29 pages, 6 figures, journal submission

Smooth convex minimization over the unit trace-norm ball is an important optimization problem in machine learning, signal processing, statistics and other fields, that underlies many tasks in which one wishes to recover a low-rank matrix given certain measurements. While first-order methods for convex optimization enjoy optimal convergence rates, they require in worst-case to compute a full-rank SVD on each iteration, in order to compute the projection onto the trace-norm ball. These full-rank SVD computations however prohibit the application of such methods to large problems. A simple and natural heuristic to reduce the computational cost is to approximate the projection using only a low-rank SVD. This raises the question if, and under what conditions, this simple heuristic can indeed result in provable convergence to the optimal solution. In this paper we show that any optimal solution is a center of a Euclid. ball inside-which the projected-gradient mapping admits rank that is at most the multiplicity of the largest singular value of the gradient vector. Moreover, the radius of the ball scales with the spectral gap of this gradient vector. We show how this readily implies the local convergence (i.e., from a "warm-start" initialization) of standard first-order methods, using only low-rank SVD computations. We also quantify the effect of "over-parameterization", i.e., using SVD computations with higher rank, on the radius of this ball, showing it can increase dramatically with moderately larger rank. We extend our results also to the setting of optimization with trace-norm regularization and optimization over bounded-trace positive semidefinite matrices. Our theoretical investigation is supported by concrete empirical evidence that demonstrates the \textit{correct} convergence of first-order methods with low-rank projections on real-world datasets., Accepted to SIAM Journal on Optimization (SIOPT)

This paper presents smoothing schemes for obtaining approximate stationary points of unconstrained or linearly-constrained composite nonconvex-concave min-max (and hence nonsmooth) problems by applying well-known algorithms to composite smooth approximations of the original problems. More specifically, in the unconstrained (resp. constrained) case, approximate stationary points of the original problem are obtained by applying, to its composite smooth approximation, an accelerated inexact proximal point (resp. quadratic penalty) method presented in a previous paper by the authors. Iteration complexity bounds for both smoothing schemes are also established. Finally, numerical results are given to demonstrate the efficiency of the unconstrained smoothing scheme.

In this work, we present a novel approach for solving stochastic shape optimization problems. Our method is the extension of the classical stochastic gradient method to infinite-dimensional shape manifolds. We prove convergence of the method on Riemannian manifolds and then make the connection to shape spaces. The method is demonstrated on a model shape optimization problem from interface identification. Uncertainty arises in the form of a random partial differential equation, where underlying probability distributions of the random coefficients and inputs are assumed to be known. We verify some conditions for convergence for the model problem and demonstrate the method numerically.

In this paper, we study an optimal control problem for a macroscopic mechanical tumour model based on the phase field approach. The model couples a Cahn--Hilliard type equation to a system of linear elasticity and a reaction-diffusion equation for a nutrient concentration. By taking advantage of previous analytical well-posedness results established by the authors, we seek optimal controls in the form of a boundary nutrient supply, as well as concentrations of cytotoxic and antiangiogenic drugs that minimise a cost functional involving mechanical stresses. Special attention is given to sparsity effects, where with the inclusion of convex non-differentiable regularisation terms to the cost functional, we can infer from the first-order optimality conditions that the optimal drug concentrations can vanish on certain time intervals., 26 pages

In this paper, we provide the first provable linear-time (in the number of non-zero entries of the input) algorithm for approximately solving the generalized trust region subproblem (GTRS) of minimizing a quadratic function over a quadratic constraint under some regular conditions. Our algorithm is motivated by and extends a recent linear-time algorithm for the trust region subproblem by Hazan and Koren [Math. Program., 2016, 158(1-2): 363-381]. However, due to the non-convexity and non-compactness of the feasible region, such an extension is nontrivial. Our main contribution is to demonstrate that under some regular condition, the optimal solution is in a compact and convex set and lower and upper bounds of the optimal value can be computed in linear time. Using these properties, we develop a linear-time algorithm for the GTRS.

We propose a methodology for studying the performance of common splitting methods through semidefinite programming. We prove tightness of the methodology and demonstrate its value by presenting two applications of it. First, we use the methodology as a tool for computer-assisted proofs to prove tight analytical contraction factors for Douglas--Rachford splitting that are likely too complicated for a human to find bare-handed. Second, we use the methodology as an algorithmic tool to computationally select the optimal splitting method parameters by solving a series of semidefinite programs., Comment: Published in the SIAM Journal on Optimization

In this paper we establish an observability inequality for the heat equation with bounded potentials on the whole space. Roughly speaking, such a kind of inequality says that the total energy of solutions can be controlled by the energy localized in a subdomain, which is equidistributed over the whole space. The proof of this inequality is mainly adapted from the parabolic frequency function method, which plays an important role in proving the unique continuation property for solutions of parabolic equations. As an immediate application, we show that the null controllability holds for the heat equation with bounded potentials on the whole space., Comment: Welcome for any comment

We address the variational formulation of the risk-sensitive reward problem for non-degenerate diffusions on $\mathbb{R}^d$ controlled through the drift. We establish a variational formula on the whole space and also show that the risk-sensitive value equals the generalized principal eigenvalue of the semilinear operator. This can be viewed as a controlled version of the variational formulas for principal eigenvalues of diffusion operators arising in large deviations. We also revisit the average risk-sensitive minimization problem and by employing a gradient estimate developed in this paper, we extend earlier results to unbounded drifts and running costs., Comment: 29 pages

Let $X,Y$ be Banach spaces, $(S_t)_{t \geq 0}$ a $C_0$-semigroup on $X$, $-A$ the corresponding infinitesimal generator on $X$, $C$ a bounded linear operator from $X$ to $Y$, and $T > 0$. We consider the system \[ \dot{x}(t) = -Ax(t), \quad y(t) = Cx(t) \quad t\in (0,T], \quad x(0) = x_0 \in X. \] We provide sufficient conditions such that this system satisfies a final state observability estimate in $L_r ((0,T) ; Y)$, $r \in [1,\infty]$. These sufficient conditions are given by an uncertainty relation and a dissipation estimate. Our approach unifies and generalizes the respective advantages from earlier results obtained in the context of Hilbert spaces. As an application we consider the example where $A$ is an elliptic operator in $L_p(\mathbb{R}^d)$ for $1

In this article, the shape optimization of a linear elastic body subject to frictional (Tresca) contact is investigated. Due to the projection operators involved in the formulation of the contact problem, the solution is not shape differentiable in general. Moreover, shape optimization of the contact zone requires the computation of the gap between the bodies in contact, as well as its shape derivative. Working with directional derivatives, sufficient conditions for shape differentiability are derived. %The problem is addressed in the general framework of two bodies with smooth boundaries. Then, some numerical results, obtained with a gradient descent algorithm based on those shape derivatives, are presented., Comment: Preprint

We analyze the coordinate descent method with a new coordinate selection strategy, called volume sampling. This strategy prescribes selecting subsets of variables of certain size proportionally to the determinants of principal submatrices of the matrix, that bounds the curvature of the objective function. In the particular case, when the size of the subsets equals one, volume sampling coincides with the well-known strategy of sampling coordinates proportionally to their Lipschitz constants. For the coordinate descent with volume sampling, we establish the convergence rates both for convex and strongly convex problems. Our theoretical results show that, by increasing the size of the subsets, it is possible to accelerate the method up to the factor which depends on the spectral gap between the corresponding largest eigenvalues of the curvature matrix. Several numerical experiments confirm our theoretical conclusions.

EXTRA is a popular method for dencentralized distributed optimization and has broad applications. This paper revisits EXTRA. First, we give a sharp complexity analysis for EXTRA with the improved $O\left(\left(\frac{L}{\mu}+\frac{1}{1-\sigma_2(W)}\right)\log\frac{1}{\epsilon(1-\sigma_2(W))}\right)$ communication and computation complexities for $\mu$-strongly convex and $L$-smooth problems, where $\sigma_2(W)$ is the second largest singular value of the weight matrix $W$. When the strong convexity is absent, we prove the $O\left(\left(\frac{L}{\epsilon}+\frac{1}{1-\sigma_2(W)}\right)\log\frac{1}{1-\sigma_2(W)}\right)$ complexities. Then, we use the Catalyst framework to accelerate EXTRA and obtain the $O\left(\sqrt{\frac{L}{\mu(1-\sigma_2(W))}}\log\frac{ L}{\mu(1-\sigma_2(W))}\log\frac{1}{\epsilon}\right)$ communication and computation complexities for strongly convex and smooth problems and the $O\left(\sqrt{\frac{L}{\epsilon(1-\sigma_2(W))}}\log\frac{1}{\epsilon(1-\sigma_2(W))}\right)$ complexities for non-strongly convex ones. Our communication complexities of the accelerated EXTRA are only worse by the factors of $\left(\log\frac{L}{\mu(1-\sigma_2(W))}\right)$ and $\left(\log\frac{1}{\epsilon(1-\sigma_2(W))}\right)$ from the lower complexity bounds for strongly convex and non-strongly convex problems, respectively.

We study the controlled dynamics of the {\it ensembles of points} of a Riemannian manifold $M$. Parameterized ensemble of points of $M$ is the image of a continuous map $\gamma:\Theta \to M$, where $\Theta$ is a compact set of parameters. The dynamics of ensembles is defined by the action $\gamma(\theta) \mapsto P_t(\gamma(\theta))$ of the semigroup of diffeomorphisms $P_t:M \to M, \ t \in \mathbb{R}$, generated by the controlled equation $\dot{x}=f(x,u(t))$ on $M$. Therefore any control system on $M$ defines a control system on (generally infinite-dimensional) space $\mathcal{E}_\Theta(M)$ of the ensembles of points. We wish to establish criteria of controllability for such control systems. As in our previous work ([1]) we seek to adapt the Lie-algebraic approach of geometric control theory to the infinite-dimensional setting. We study the case of finite ensembles and prove genericity of exact controllability property for them. We also find sufficient approximate controllability criterion for continual ensembles and prove a result on motion planning in the space of flows on $M$. We discuss the relation of the obtained controllability criteria to various versions of Rashevsky-Chow theorem for finite- and infinite-dimensional manifolds., Comment: 24 pages

We study the $\textit{Short-and-Sparse (SaS) deconvolution}$ problem of recovering a short signal $\mathbf a_0$ and a sparse signal $\mathbf x_0$ from their convolution. We propose a method based on nonconvex optimization, which under certain conditions recovers the target short and sparse signals, up to a signed shift symmetry which is intrinsic to this model. This symmetry plays a central role in shaping the optimization landscape for deconvolution. We give a $\textit{regional analysis}$, which characterizes this landscape geometrically, on a union of subspaces. Our geometric characterization holds when the length-$p_0$ short signal $\mathbf a_0$ has shift coherence $\mu$, and $\mathbf x_0$ follows a random sparsity model with sparsity rate $\theta \in \Bigl[\frac{c_1}{p_0}, \frac{c_2}{p_0\sqrt\mu + \sqrt{p_0}}\Bigr]\cdot\frac{1}{\log^2p_0}$. Based on this geometry, we give a provable method that successfully solves SaS deconvolution with high probability.

In this paper we consider the unconstrained minimization problem of a smooth function in ${\mathbb{R}}^n$ in a setting where only function evaluations are possible. We design a novel randomized derivative-free algorithm --- the stochastic three points (STP) method --- and analyze its iteration complexity. At each iteration, STP generates a random search direction according to a certain fixed probability law. Our assumptions on this law are very mild: roughly speaking, all laws which do not concentrate all measure on any halfspace passing through the origin will work. For instance, we allow for the uniform distribution on the sphere and also distributions that concentrate all measure on a positive spanning set. Given a current iterate $x$, STP compares the objective function at three points: $x$, $x+\alpha s$ and $x-\alpha s$, where $\alpha>0$ is a stepsize parameter and $s$ is the random search direction. The best of these three points is the next iterate. We analyze the method STP under several stepsize selection schemes (fixed, decreasing, estimated through finite differences, etc). We study non-convex, convex and strongly convex cases.

In this paper, we propose new accelerated methods for smooth convex optimization, called contracting proximal methods. At every step of these methods, we need to minimize a contracted version of the objective function augmented by a regularization term in the form of Bregman divergence. We provide global convergence analysis for a general scheme admitting inexactness in solving the auxiliary subproblem. In the case of using for this purpose high-order tensor methods, we demonstrate an acceleration effect for both convex and uniformly convex composite objective functions. Thus, our construction explains acceleration for methods of any order starting from one. The augmentation of the number of calls of oracle due to computing the contracted proximal steps is limited by the logarithmic factor in the worst-case complexity bound.

International audience; We are concerned with the design of Model Predictive Control (MPC) schemes such that asymptotic stability of the resulting closed loop is guaranteed even if the linearization at the desired set point fails to be stabilizable. Therefore, we propose to construct the stage cost based on the homogeneous approximation and rigorously show that applying MPC yields an asymp-totically stable closed-loop behavior if the homogeneous approximation is asymptotically null controllable. To this end, we verify cost controllability-a condition relating the current state, the stage cost, and the growth of the value function w.r.t. time-for this class of systems in order to provide stability and performance guarantees for the proposed MPC scheme without stabilizing terminal costs or constraints.

International audience; Obtaining sharp Lipschitz constants for feed-forward neural networks is essential to assess their robust-ness in the face of perturbations of their inputs. We derive such constants in the context of a general layered network model involving compositions of nonexpansive averaged operators and affine operators. By exploiting this architecture, our analysis finely captures the interactions between the layers, yielding tighter Lipschitz constants than those resulting from the product of individual bounds for groups of layers. The proposed framework is shown to cover in particular most practical instances encountered in feed-forward neural networks. Our Lipschitz constant estimates are further improved in the case of structures employing scalar nonlinear functions, which include standard convolutional networks as special cases.

In this paper we study $p$-order methods for unconstrained minimization of convex functions that are $p$-times differentiable ($p\geq 2$) with $\nu$-H\"{o}lder continuous $p$th derivatives. We propose tensor schemes with and without acceleration. For the schemes without acceleration, we establish iteration complexity bounds of $\mathcal{O}\left(\epsilon^{-1/(p+\nu-1)}\right)$ for reducing the functional residual below a given $\epsilon\in (0,1)$. Assuming that $\nu$ is known, we obtain an improved complexity bound of $\mathcal{O}\left(\epsilon^{-1/(p+\nu)}\right)$ for the corresponding accelerated scheme. For the case in which $\nu$ is unknown, we present a universal accelerated tensor scheme with iteration complexity of $\mathcal{O}\left(\epsilon^{-p/[(p+1)(p+\nu-1)]}\right)$. A lower complexity bound of $\mathcal{O}\left(\epsilon^{-2/[3(p+\nu)-2]}\right)$ is also obtained for this problem class., Comment: arXiv admin note: text overlap with arXiv:1907.07053

We consider distributed smooth nonconvex unconstrained optimization over networks, modeled as a connected graph. We examine the behavior of distributed gradient-based algorithms near strict saddle points. Specifically, we establish that (i) the renowned Distributed Gradient Descent (DGD) algorithm likely converges to a neighborhood of a Second-order Stationary (SoS) solution; and (ii) the more recent class of distributed algorithms based on gradient tracking--implementable also over digraphs--likely converges to exact SoS solutions, thus avoiding (strict) saddle-points. Furthermore, new convergence rate results to first-order critical points is established for the latter class of algorithms., Comment: Final version, to appear on SIAM J. on Optimization

For the first time, this paper investigates the phase retrieval problem with the assumption that the phase (of the complex signal) is sparse in contrast to the sparsity assumption on the signal itself as considered in the literature of sparse signal processing. The intended application of this new problem model, which will be conducted in a follow-up paper, is to practical phase retrieval problems where the aberration phase is sparse with respect to the orthogonal basis of Zernike polynomials. Such a problem is called sparse phase retrieval (SPR) problem in this paper. When the amplitude modulation at the exit pupil is uniform, a new scheme of sparsity regularization on phase is proposed to capture the sparsity property of the SPR problem. Based on this regularization scheme, we design and analyze an efficient solution method, named SROP algorithm, for solving SPR given only a single intensity point-spread-function image. The algorithm is a combination of the Gerchberg-Saxton algorithm with the newly proposed sparsity regularization on the phase. The latter regularization step is mathematically a rotation but with direction varying in iterations. Surprisingly, this rotation is shown to be a metric projection on an auxiliary set which is independent of iterations. As a consequence, SROP algorithm is proved to be the cyclic projections algorithm for solving a feasibility problem involving three auxiliary sets. Analyzing regularity properties of the latter auxiliary sets, we obtain convergence results for SROP algorithm based on recent convergence theory for the cyclic projections algorithm. Numerical results show clear effectiveness of the new regularization scheme for solving the SPR problem.

For sequential stochastic control problems with standard Borel measurement and control action spaces, we introduce a general (universally applicable) dynamic programming formulation, establish its well-posedness, and provide new existence results for optimal policies. Our dynamic program builds in part on Witsenhausen's standard form, but with a different formulation for the state, action, and transition dynamics. Using recent results on measurability properties of strategic measures in decentralized control, we obtain a standard Borel controlled Markov model. This allows for a well-defined dynamic programming recursion through universal measurability properties of the value functions for each time stage. In addition, new existence results are obtained for optimal policies in decentralized stochastic control. These state that for a static team with independent measurements, it suffices for the cost function to be continuous (only) in the actions for the existence of an optimal policy under mild compactness (or tightness) conditions. These also apply to dynamic teams which admit static reductions with independent measurements through a change of measure transformation. We show through a counterexample that weaker conditions may not lead to existence of an optimal team policy. The paper's existence results generalize those previously reported in the literature. A summary of and comparison with previously reported results and some applications are presented., To appear in SIAM Journal on Control and Optimization

We address the variational problem for the generalized principal eigenvalue on $\mathbb{R}^d$ of linear and semilinear elliptic operators associated with nondegenerate diffusions controlled through the drift. We establish the Collatz-Wielandt formula for potentials that vanish at infinity under minimal hypotheses, and also for general potentials under blanket geometric ergodicity assumptions. We also present associated results having the flavor of a refined maximum principle., Comment: 19 pages

In this paper we study a bilinear optimal control problem associated to a 3D chemo-repulsion model with linear production. We prove the existence of weak solutions and we establish a regularity criterion to get global in time strong solutions. As a consequence, we deduce the existence of a global optimal solution with bilinear control and, using a Lagrange multipliers theorem, we derive first-order optimality conditions for local optimal solutions., 41 pages. arXiv admin note: text overlap with arXiv:1806.10076

We build upon Estrin et al. (2019) to develop a general constrained nonlinear optimization algorithm based on a smooth penalty function proposed by Fletcher (1970, 1973b). Although Fletcher's approach has historically been considered impractical, we show that the computational kernels required are no more expensive than those in other widely accepted methods for nonlinear optimization. The main kernel for evaluating the penalty function and its derivatives solves structured linear systems. When the matrices are available explicitly, we store a single factorization each iteration. Otherwise, we obtain a factorization-free optimization algorithm by solving each linear system iteratively. The penalty function shows promise in cases where the linear systems can be solved efficiently, e.g., PDE-constrained optimization problems when efficient preconditioners exist. We demonstrate the merits of the approach, and give numerical results on several PDE-constrained and standard test problems., 25 pages. arXiv admin note: text overlap with arXiv:1910.04300