Search

Following the breakthrough work of Tardos (Oper Res 34:250–256, 1986) in the bit-complexity model, Vavasis and Ye (Math Program 74(1):79–120, 1996) gave the first exact algorithm for linear programming in the real model of computation with running time depending only on the constraint matrix. For solving a linear program (LP) $$\max \, c^\top x,\, Ax = b,\, x \ge 0,\, A \in \mathbb {R}^{m \times n}$$ max c ⊤ x , A x = b , x ≥ 0 , A ∈ R m × n , Vavasis and Ye developed a primal-dual interior point method using a ‘layered least squares’ (LLS) step, and showed that $$O(n^{3.5} \log (\bar{\chi }_A+n))$$ O ( n 3.5 log ( χ ¯ A + n ) ) iterations suffice to solve (LP) exactly, where $$\bar{\chi }_A$$ χ ¯ A is a condition measure controlling the size of solutions to linear systems related to A. Monteiro and Tsuchiya (SIAM J Optim 13(4):1054–1079, 2003), noting that the central path is invariant under rescalings of the columns of A and c, asked whether there exists an LP algorithm depending instead on the measure $$\bar{\chi }^*_A$$ χ ¯ A ∗ , defined as the minimum $$\bar{\chi }_{AD}$$ χ ¯ AD value achievable by a column rescaling AD of A, and gave strong evidence that this should be the case. We resolve this open question affirmatively. Our first main contribution is an $$O(m^2 n^2 + n^3)$$ O ( m 2 n 2 + n 3 ) time algorithm which works on the linear matroid of A to compute a nearly optimal diagonal rescaling D satisfying $$\bar{\chi }_{AD} \le n(\bar{\chi }_A^*)^3$$ χ ¯ AD ≤ n ( χ ¯ A ∗ ) 3 . This algorithm also allows us to approximate the value of $$\bar{\chi }_A$$ χ ¯ A up to a factor $$n (\bar{\chi }_A^*)^2$$ n ( χ ¯ A ∗ ) 2 . This result is in surprising contrast to that of Tunçel (Math Program 86(1):219–223, 1999), who showed NP-hardness for approximating $$\bar{\chi }_A$$ χ ¯ A to within $$2^{\textrm{poly}(\textrm{rank}(A))}$$ 2 poly ( rank ( A ) ) . The key insight for our algorithm is to work with ratios $$g_i/g_j$$ g i / g j of circuits of A—i.e., minimal linear dependencies $$Ag=0$$ A g = 0 —which allow us to approximate the value of $$\bar{\chi }_A^*$$ χ ¯ A ∗ by a maximum geometric mean cycle computation in what we call the ‘circuit ratio digraph’ of A. While this resolves Monteiro and Tsuchiya’s question by appropriate preprocessing, it falls short of providing either a truly scaling invariant algorithm or an improvement upon the base LLS analysis. In this vein, as our second main contribution we develop a scaling invariant LLS algorithm, which uses and dynamically maintains improving estimates of the circuit ratio digraph, together with a refined potential function based analysis for LLS algorithms in general. With this analysis, we derive an improved $$O(n^{2.5} \log (n)\log (\bar{\chi }^*_A+n))$$ O ( n 2.5 log ( n ) log ( χ ¯ A ∗ + n ) ) iteration bound for optimally solving (LP) using our algorithm. The same argument also yields a factor $$n/\log n$$ n / log n improvement on the iteration complexity bound of the original Vavasis–Ye algorithm.

International audience; This paper addresses a large class of vector optimization problems in infinite-dimensional spaces with respect to two important binary relations derived from domination structures. Motivated by theoretical challenges as well as by applications to some models in behavioral sciences, we establish new variational principles that can be viewed as far-going extensions of the Ekeland variational principle to cover domination vector settings. Our approach combines advantages of both primal and dual techniques in variational analysis with providing useful sufficient conditions for the existence of variational traps in behavioral science models with variable domination structures.

The problem of finding a best approximation pair of two sets, which in turn generalizes the well known convex feasibility problem, has a long history that dates back to work by Cheney and Goldstein in 1959. In 2018, Aharoni, Censor, and Jiang revisited this problem and proposed an algorithm that can be used when the two sets are finite intersections of halfspaces. Motivated by their work, we present alternative algorithms that utilize projection and proximity operators. Our modeling framework is able to accommodate even convex sets. Numerical experiments indicate that these methods are competitive and sometimes superior to the one proposed by Aharoni et al.

The notion of best approximation mapping (BAM) with respect to a closed affine subspace in finite-dimensional space was introduced by Behling, Bello Cruz and Santos to show the linear convergence of the block-wise circumcentered-reflection method. The best approximation mapping possesses two critical properties of the circumcenter mapping for linear convergence. Because the iteration sequence of BAM linearly converges, the BAM is interesting in its own right. In this paper, we naturally extend the definition of BAM from closed affine subspace to nonempty closed convex set and from $\mathbb{R}^{n}$ to general Hilbert space. We discover that the convex set associated with the BAM must be the fixed point set of the BAM. Hence, the iteration sequence generated by a BAM linearly converges to the nearest fixed point of the BAM. Connections between BAMs and other mappings generating convergent iteration sequences are considered. Behling et al.\ proved that the finite composition of BAMs associated with closed affine subspaces is still a BAM in $\mathbb{R}^{n}$. We generalize their result from $\mathbb{R}^{n}$ to general Hilbert space and also construct a new constant associated with the composition of BAMs. This provides a new proof of the linear convergence of the method of alternating projections. Moreover, compositions of BAMs associated with general convex sets are investigated. In addition, we show that convex combinations of BAMs associated with affine subspaces are BAMs. Last but not least, we connect BAM with circumcenter mapping in Hilbert spaces., 31 pages and 2 figures

Circumcenters play an important role in the design and analysis of accelerating various iterative methods in optimization. In this work, we propose Bregman (pseudo-)circumcenters associated with finite sets. We show the existence and give explicit formulae for the unique backward and forward Bregman pseudo-circumcenters of finite sets. Moreover, we use duality to establish connections between backward and forward Bregman (pseudo-)circumcenters. Various examples are presented to illustrate the backward and forward Bregman (pseudo-)circumcenters of finite sets. Our general framework for circumcenters paves the way for the development of accelerating iterative methods by Bregman circumcenters., Comment: 24 pages and 4 figures

In this work, we develop a systematic framework for computing the resolvent of the sum of two or more monotone operators which only activates each operator in the sum individually. The key tool in the development of this framework is the notion of the “strengthening” of a set-valued operator, which can be viewed as a type of regularisation that preserves computational tractability. After deriving a number of iterative schemes through this framework, we demonstrate their application to best approximation problems, image denoising and elliptic PDEs. FJAA and RC were partially supported by the Ministry of Science, Innovation and Universities of Spain and the European Regional Development Fund (ERDF) of the European Commission, Grant PGC2018-097960-B-C22. MKT is supported in part by ARC grant DE200100063.

Given integers n $\geq$ m, let Sep(n,m) be the set of separable states on the Hilbert space $\mathbb{C}^n \otimes \mathbb{C}^m$. It is well-known that for (n,m)=(3,2) the set of separable states has a simple description using semidefinite programming: it is given by the set of states that have a positive partial transpose. In this paper we show that for larger values of n and m the set Sep(n,m) has no semidefinite programming description of finite size. As Sep(n,m) is a semialgebraic set this provides a new counterexample to the Helton-Nie conjecture, which was recently disproved by Scheiderer in a breakthrough result. Compared to Scheiderer's approach, our proof is elementary and relies only on basic results about semialgebraic sets and functions., 16 pages

Gradient-based optimization algorithms can be studied from the perspective of limiting ordinary differential equations (ODEs). Motivated by the fact that existing ODEs do not distinguish between two fundamentally different algorithms---Nesterov's accelerated gradient method for strongly convex functions (NAG-SC) and Polyak's heavy-ball method---we study an alternative limiting process that yields high-resolution ODEs. We show that these ODEs permit a general Lyapunov function framework for the analysis of convergence in both continuous and discrete time. We also show that these ODEs are more accurate surrogates for the underlying algorithms; in particular, they not only distinguish between NAG-SC and Polyak's heavy-ball method, but they allow the identification of a term that we refer to as "gradient correction" that is present in NAG-SC but not in the heavy-ball method and is responsible for the qualitative difference in convergence of the two methods. We also use the high-resolution ODE framework to study Nesterov's accelerated gradient method for (non-strongly) convex functions, uncovering a hitherto unknown result---that NAG-C minimizes the squared gradient norm at an inverse cubic rate. Finally, by modifying the high-resolution ODE of NAG-C, we obtain a family of new optimization methods that are shown to maintain the accelerated convergence rates of NAG-C for smooth convex functions., 82 pages, 11 figures

We address the problem of prescribing an optimal decision in a framework where the cost function depends on uncertain problem parameters that need to be learned from data. Earlier work proposed prescriptive formulations based on supervised machine learning methods. These prescriptive methods can factor in contextual information on a potentially large number of covariates to take context specific actions which are superior to any static decision. When working with noisy or corrupt data, however, such nominal prescriptive methods can be prone to adverse overfitting phenomena and fail to generalize on out-of-sample data. In this paper we combine ideas from robust optimization and the statistical bootstrap to propose novel prescriptive methods which safeguard against overfitting. We show indeed that a particular entropic robust counterpart to such nominal formulations guarantees good performance on synthetic bootstrap data. As bootstrap data is often a sensible proxy to actual out-of-sample data, our robust counterpart can be interpreted to directly encourage good out-of-sample performance. The associated robust prescriptive methods furthermore reduce to convenient tractable convex optimization problems in the context of local learning methods such as nearest neighbors and Nadaraya-Watson learning. We illustrate our data-driven decision-making framework and our novel robustness notion on a small newsvendor problem.

In this paper, we give an overview on optimality conditions and exact penalization for the mathematical program with switching constraints (MPSC). MPSC is a new class of optimization problems which has some important applications. It is well-known that if MPSC is treated as a standard nonlinear program, some of the usual constraint qualifications may fail and to deal with this issue one could reformulate it as a mathematical program with disjunctive constraints (MPDC). In this paper we first survey recent results on constraint qualifications and optimality conditions for MPDC and then apply them to MPSC to obtain the corresponding constraint qualifications and optimality conditions. Moreover we provide two types of sufficient conditions for the local error bound and exact penalty results for MPSC. One comes from the directional quasi-normality for MPDC and the other is obtained by using the local decomposition approach.

We study a first-order primal-dual subgradient method to optimize risk-constrained risk-penalized optimization problems, where risk is modeled via the popular conditional value at risk (CVaR) measure. The algorithm processes independent and identically distributed samples from the underlying uncertainty in an online fashion, and produces an $\eta/\sqrt{K}$-approximately feasible and $\eta/\sqrt{K}$-approximately optimal point within $K$ iterations with constant step-size, where $\eta$ increases with tunable risk-parameters of CVaR. We find optimized step sizes using our bounds and precisely characterize the computational cost of risk aversion as revealed by the growth in $\eta$. Our proposed algorithm makes a simple modification to a typical primal-dual stochastic subgradient algorithm. With this mild change, our analysis surprisingly obviates the need for a priori bounds or complex adaptive bounding schemes for dual variables assumed in many prior works. We also draw interesting parallels in sample complexity with that for chance-constrained programs derived in the literature with a very different solution architecture., Comment: 30 pages, 2 figures; Revised in order to formalize relation to existing primal-dual algorithms, better emphasize the salient features of the algorithm, and update the example

Subgradient and Newton algorithms for nonsmooth optimization require generalized derivatives to satisfy subtle approximation properties: conservativity for the former and semismoothness for the latter. Though these two properties originate in entirely different contexts, we show that in the semi-algebraic setting they are equivalent. Both properties for a generalized derivative simply require it to coincide with the standard directional derivative on the tangent spaces of some partition of the domain into smooth manifolds. An appealing byproduct is a new short proof that semi-algebraic maps are semismooth relative to the Clarke Jacobian., 12 pages

Wasserstein balls, which contain all probability measures within a pre-specified Wasserstein distance to a reference measure, have recently enjoyed wide popularity in the distributionally robust optimization and machine learning communities to formulate and solve data-driven optimization problems with rigorous statistical guarantees. In this technical note we prove that the Wasserstein ball is weakly compact under mild conditions, and we offer necessary and sufficient conditions for the existence of optimal solutions. We also characterize the sparsity of solutions if the Wasserstein ball is centred at a discrete reference measure. In comparison with the existing literature, which has proved similar results under different conditions, our proofs are self-contained and shorter, yet mathematically rigorous, and our necessary and sufficient conditions for the existence of optimal solutions are easily verifiable in practice.

Many iterative methods in applied mathematics can be thought of as fixed-point iterations, and such algorithms are usually analyzed analytically, with inequalities. In this paper, we present a geometric approach to analyzing contractive and nonexpansive fixed point iterations with a new tool called the scaled relative graph (SRG). The SRG provides a correspondence between nonlinear operators and subsets of the 2D plane. Under this framework, a geometric argument in the 2D plane becomes a rigorous proof of convergence., Published in Mathematical Programming

The main focus in this paper is exact linesearch methods for minimizing a quadratic function whose Hessian is positive definite. We give a class of limited-memory quasi-Newton Hessian approximations which generate search directions parallel to those of the BFGS method, or equivalently, to those of the method of preconditioned conjugate gradients. In the setting of reduced Hessians, the class provides a dynamical framework for the construction of limited-memory quasi-Newton methods. These methods attain finite termination on quadratic optimization problems in exact arithmetic. We show performance of the methods within this framework in finite precision arithmetic by numerical simulations on sequences of related systems of linear equations, which originate from the CUTEst test collection. In addition, we give a compact representation of the Hessian approximations in the full Broyden class for the general unconstrained optimization problem. This representation consists of explicit matrices and gradients only as vector components.

In this paper we consider an impulsive extension of an optimal control problem with unbounded controls, subject to endpoint and state constraints. We show that the existence of an extended-sense minimizer that is a normal extremal for a constrained Maximum Principle ensures that there is no gap between the infima of the original problem and of its extension. Furthermore, we translate such relation into verifiable sufficient conditions for normality in the form of constraint and endpoint qualifications. Links between existence of an infimum gap and normality in impulsive control have previously been explored for problems without state constraints. This paper establishes such links in the presence of state constraints and of an additional ordinary control, for locally Lipschitz continuous data.

This paper discusses an outer-approximation guided optimization method for constrained neural network inverse problems with rectified linear units. The constrained neural network inverse problems refer to an optimization problem to find the best set of input values of a given trained neural network in order to produce a predefined desired output in presence of constraints on input values. This paper analyzes the characteristics of optimal solutions of neural network inverse problems with rectified activation units and proposes an outer-approximation algorithm by exploiting their characteristics. The proposed outer-approximation guided optimization comprises primal and dual phases. The primal phase incorporates neighbor curvatures with neighbor outer-approximations to expedite the process. The dual phase identifies and utilizes the structure of local convex regions to improve the convergence to a local optimal solution. At last, computation experiments demonstrate the superiority of the proposed algorithm compared to a projected gradient method.

This work provides analysis of a variant of the Risk-Sharing Principal-Agent problem in a single period setting with additional constant lower and upper bounds on the wage paid to the Agent. First the effect of the extra constraints on optimal contract existence is analyzed and leads to conditions on utilities under which an optimum may be attained. Solution characterization is then provided along with the derivation of a Borch rule for Limited Liability. Finally the CARA utility case is considered and a closed form optimal wage and action are obtained. This allows for analysis of the classical CARA utility and gaussian setting.

This paper considers a general convex constrained problem setting where functions are not assumed to be differentiable nor Lipschitz continuous. Our motivation is in finding a simple first-order method for solving a wide range of convex optimization problems with minimal requirements. We study the method of weighted dual averages (Nesterov in Math Programm 120(1): 221–259, 2009) in this setting and prove that it is an optimal method.

Certifying function nonnegativity is a ubiquitous problem in computational mathematics, with especially notable applications in optimization. We study the question of certifying nonnegativity of signomials based on the recently proposed approach of Sums-of-AM/GM-Exponentials (SAGE) decomposition due to the second author and Shah. The existence of a SAGE decomposition is a sufficient condition for nonnegativity of a signomial, and it can be verified by solving a tractable convex relative entropy program. We present new structural properties of SAGE certificates such as a characterization of the extreme rays of the cones associated to these decompositions as well as an appealing form of sparsity preservation. These lead to a number of important consequences such as conditions under which signomial nonnegativity is equivalent to the existence of a SAGE decomposition; our results represent the broadest-known class of nonconvex signomial optimization problems that can be solved efficiently via convex relaxation. The analysis in this paper proceeds by leveraging the interaction between the convex duality underlying SAGE certificates and the face structure of Newton polytopes. While our primary focus is on signomials, we also discuss how our results provide efficient methods for certifying polynomial nonnegativity, with complexity independent of the degree of a polynomial., Body shortened from 29 to 24 pages. Additional consideration to related work. Some claims made in Section 5 have been formalized. Revised within 2 months of first-round reviews

In a first contribution, we revisit two certificates of positivity on (possibly non-compact) basic semialgebraic sets due to Putinar and Vasilescu [Comptes Rendus de l'Acad\'emie des Sciences-Series I-Mathematics, 328(6) (1999) pp. 495-499]. We use Jacobi's technique from [Mathematische Zeitschrift, 237(2) (2001) pp. 259-273] to provide an alternative proof with an effective degree bound on the sums of squares multipliers in such certificates. As a consequence, it allows one to define a hierarchy of semidefinite relaxations for a general polynomial optimization problem. Convergence of this hierarchy to a neighborhood of the optimal value as well as strong duality and analysis are guaranteed. In a second contribution, we introduce a new numerical method for solving systems of polynomial inequalities and equalities with possibly uncountably many solutions. As a bonus, one may apply this method to obtain approximate global optimizers in polynomial optimization., Comment: 33 pages, 2 figures, 5 tables

The discrete Wasserstein barycenter problem is a minimum-cost mass transport problem for a set of probability measures with finite support. In this paper, we show that finding a barycenter of sparse support is hard, even in dimension 2 and for only 3 measures. We prove this claim by showing that a special case of an intimately related decision problem SCMP—does there exist a measure with a non-mass-splitting transport cost and support size below prescribed bounds? Is NP-hard for all rational data. Our proof is based on a reduction from planar 3-dimensional matching and follows a strategy laid out by Spieksma and Woeginger (Eur J Oper Res 91:611–618, 1996) for a reduction to planar, minimum circumference 3-dimensional matching. While we closely mirror the actual steps of their proof, the arguments themselves differ fundamentally due to the complex nature of the discrete barycenter problem. Containment of SCMP in NP will remain open. We prove that, for a given measure, sparsity and cost of an optimal transport to a set of measures can be verified in polynomial time in the size of a bit encoding of the measure. However, the encoding size of a barycenter may be exponential in the encoding size of the underlying measures.

We propose finitely convergent methods for solving convex feasibility problems defined over a possibly infinite pool of constraints. Following other works in this area, we assume that the interior of the solution set is nonempty and that certain overrelaxation parameters form a divergent series. We combine our methods with a very general class of deterministic control sequences where, roughly speaking, we require that sooner or later we encounter a violated constraint if one exists. This requirement is satisfied, in particular, by the cyclic, repetitive and remotest set controls. Moreover, it is almost surely satisfied for random controls.

This paper studies data-driven distributionally robust bottleneck combinatorial problems (DRBCP) with stochastic costs, where the probability distribution of the cost vector is contained in a ball of distributions centered at the empirical distribution specified by the Wasserstein distance. We study two distinct versions of DRBCP from different applications: (i) Motivated by the multi-hop wireless network application, we first study the uncertainty quantification of DRBCP (denoted by DRBCP-U), where decision-makers would like to have an accurate estimation of the worst-case value of DRBCP. The difficulty of DRBCP-U is to handle its max-min-max form. Fortunately, the alternative forms of the bottleneck combinatorial problems from their blockers allow us to derive equivalent deterministic reformulations, which can be computed via mixed-integer programs. In addition, by drawing the connection between DRBCP-U and its sampling average approximation counterpart under empirical distribution, we show that the Wasserstein radius can be chosen in the order of negative square root of sample size, improving the existing known results; and (ii) Next, motivated by the ride-sharing application, decision-makers choose the best service-and-passenger matching that minimizes the unfairness. This gives rise to the decision-making DRBCP (denoted by DRBCP-D). For DRBCP-D, we show that its optimal solution is also optimal to its sampling average approximation counterpart, and the Wasserstein radius can be chosen in a similar order as DRBCP-U. When the sample size is small, we propose to use the optimal value of DRBCP-D to construct an indifferent solution space and propose an alternative decision-robust model, which finds the best indifferent solution to minimize the empirical variance. We further show that the decision robust model can be recast as a mixed-integer program., 32 pages, 4 figures

The network reconfiguration problem seeks to find a rooted tree $T$ such that the energy of the (unique) feasible electrical flow over $T$ is minimized. The tree requirement on the support of the flow is motivated by operational constraints in electricity distribution networks. The bulk of existing results on convex optimization over vertices of polytopes and on the structure of electrical flows do not easily give guarantees for this problem, while many heuristic methods have been developed in the power systems community as early as 1989. Our main contribution is to give the first provable approximation guarantees for the network reconfiguration problem. We provide novel lower bounds and corresponding approximation factors for various settings ranging from $\min\{O(m-n), O(n)\}$ for general graphs, to $O(\sqrt{n})$ over grids with uniform resistances on edges, and $O(1)$ for grids with uniform edge resistances and demands. To obtain the result for general graphs, we propose a new method for (approximate) spectral graph sparsification, which may be of independent interest. Using insights from our theoretical results, we propose a general heuristic for the network reconfiguration problem that is orders of magnitude faster than existing methods in the literature, while obtaining comparable performance., 37 pages, 11 figures

In the Target-Attacker-Defender (TAD) differential game, an Attacker missile strives to capture a Target aircraft. The Target tries to escape the Attacker and is aided by a Defender missile which aims at intercepting the Attacker before the latter manages to close in on the Target. The conflict between these intelligent adversaries has been suitably modeled as a zero-sum differential game. Optimal strategies have been synthesized covering the region of the state space where the Target/Defender team is able to win the game. However, the Game of Degree in the Attacker's region of win has not been fully addressed. Preliminary attempts at designing the players' strategies have not been proven to be optimal in the differential game sense. The main results of the paper present the optimal strategies of the Game of Degree in the Attacker's winning region of the state space. It is proven that the obtained strategies provide the saddle-point solution of the game; the Value function is obtained and it is shown to be continuous and continuously differentiable. It is also demonstrated that it is the solution of the Hamilton-Jacobi-Isaacs (HJI) equation. Finally, the obtained strategies are compared to recent results addressing the TAD differential game in [22]. It is shown by counterexample that the strategies proposed in [22] are not optimal. The unique regular solution of this differential game that actually provides a semipermeable Barrier surface is synthesized and verified in this paper., Comment: 12 pages, 6 figures

Using polarity, we give an outer polyhedral approximation for the epigraph of set functions. For a submodular function, we prove that the corresponding polar relaxation is exact; hence, it is equivalent to the Lov\'asz extension. The polar approach provides an alternative proof for the convex hull description of the epigraph of a submodular function. Computational experiments show that the inequalities from outer approximations can be effective as cutting planes for solving submodular as well as non-submodular set function minimization problems.

Given polynomial maps $f, g \colon \mathbb {R}^{n} \to \mathbb {R}^{n}$ , we consider the polynomial complementary problem of finding a vector $x \in \mathbb {R}^{n}$ such that $$ f(x) \ge 0, \quad g(x) \ge 0, \quad \text{ and } \quad \langle f(x), g(x) \rangle = 0. $$ In this paper, we present various properties on the solution set of the problem, including genericity, nonemptiness, compactness, uniqueness as well as error bounds with explicit exponents. These strengthen and generalize some previously known results.

The Nemhauser-Trotter theorem states that the standard linear programming (LP) formulation for the stable set problem has a remarkable property, also known as (weak) persistency: for every optimal LP solution that assigns integer values to some variables, there exists an optimal integer solution in which these variables retain the same values. While the standard LP is defined by only non-negativity and edge constraints, a variety of other LP formulations have been studied and one may wonder whether any of them has this property as well. We show that any other formulation that satisfies mild conditions cannot have the persistency property on all graphs, unless it is always equal to the stable set polytope., Comment: 17 pages, 6 figures

Sentence compression is an important problem in natural language processing with wide applications in text summarization, search engine and human-AI interaction system etc. In this paper, we design a hybrid extractive sentence compression model combining a probability language model and a parse tree language model for compressing sentences by guaranteeing the syntax correctness of the compression results. Our compression model is formulated as an integer linear programming problem, which can be rewritten as a Difference-of-Convex (DC) programming problem based on the exact penalty technique. We use a well-known efficient DC algorithm -- DCA to handle the penalized problem for local optimal solutions. Then a hybrid global optimization algorithm combining DCA with a parallel branch-and-bound framework, namely PDCABB, is used for finding global optimal solutions. Numerical results demonstrate that our sentence compression model can provide excellent compression results evaluated by F-score, and indicate that PDCABB is a promising algorithm for solving our sentence compression model., Full length paper (a short version of this paper for conference proceedings can be found arXiv:1902.07248)

The Karush-Kuhn-Tucker and value function (lower-level value function, to be precise) reformulations are the most common single-level transformations of the bilevel optimization problem. So far, these reformulations have either been studied independently or as a joint optimization problem in an attempt to take advantage of the best properties from each model. To the best of our knowledge, these reformulations have not yet been compared in the existing literature. This paper is a first attempt towards establishing whether one of these reformulations is best at solving a given class of the optimistic bilevel optimization problem. We design a comparison framework, which seems fair, considering the theoretical properties of these reformulations. This work reveals that although none of the models seems to particularly dominate the other from the theoretical point of view, the value function reformulation seems to numerically outperform the Karush-Kuhn-Tucker reformulation on a Newton-type algorithm. The computational experiments here are mostly based on test problems from the Bilevel Optimization LIBrary (BOLIB)., 33 pages, 4 figures

We show that, if $\mathbf{W} \in \mathbb{R}^{N \times N}_{\mathsf{sym}}$ is drawn from the gaussian orthogonal ensemble, then with high probability the degree 4 sum-of-squares relaxation cannot certify an upper bound on the objective $N^{-1} \cdot \mathbf{x}^\top \mathbf{W} \mathbf{x}$ under the constraints $x_i^2 - 1 = 0$ (i.e. $\mathbf{x} \in \{ \pm 1 \}^N$) that is asymptotically smaller than $\lambda_{\max}(\mathbf{W}) \approx 2$. We also conjecture a proof technique for lower bounds against sum-of-squares relaxations of any degree held constant as $N \to \infty$, by proposing an approximate pseudomoment construction., Comment: 34 pages. Minor text revisions; closest to published version to appear in Mathematical Programming

In the present paper, it is provided a representation result for the weak solutions of a class of evolutionary Hamilton-Jacobi-Bellman equations on infinite horizon, with Hamiltonians measurable in time and fiber convex. Such Hamiltonians are associated with a - faithful - representation, namely involving two functions measurable in time and locally Lipschitz in the state and control. Our results concern the recovering of a representation of convex Hamiltonians under a relaxed assumption on the Fenchel transform of the Hamiltonian with respect to the fiber. We apply them to investigate the uniqueness of weak solutions, vanishing at infinity, of a class of time-dependent Hamilton-Jacobi-Bellman equations. Assuming a viability condition on the domain of the aforementioned Fenchel transform, these weak solutions are regarded as an appropriate value function of an infinite horizon control problem under state constraints.

We consider the generalized trust region subproblem (GTRS) of minimizing a nonconvex quadratic objective over a nonconvex quadratic constraint. A lifting of this problem recasts the GTRS as minimizing a linear objective subject to two nonconvex quadratic constraints. Our first main contribution is structural: we give an explicit description of the convex hull of this nonconvex set in terms of the generalized eigenvalues of an associated matrix pencil. This result may be of interest in building relaxations for nonconvex quadratic programs. Moreover, this result allows us to reformulate the GTRS as the minimization of two convex quadratic functions in the original space. Our next set of contributions is algorithmic: we present an algorithm for solving the GTRS up to an $$\epsilon $$ additive error based on this reformulation. We carefully handle numerical issues that arise from inexact generalized eigenvalue and eigenvector computations and establish explicit running time guarantees for these algorithms. Notably, our algorithms run in linear (in the size of the input) time. Furthermore, our algorithm for computing an $$\epsilon $$ -optimal solution has a slightly-improved running time dependence on $$\epsilon $$ over the state-of-the-art algorithm. Our analysis shows that the dominant cost in solving the GTRS lies in solving a generalized eigenvalue problem—establishing a natural connection between these problems. Finally, generalizations of our convex hull results allow us to apply our algorithms and their theoretical guarantees directly to equality-, interval-, and hollow-constrained variants of the GTRS. This gives the first linear-time algorithm in the literature for these variants of the GTRS.

We consider an $\ell_0$-minimization problem where $f(x) + \gamma \|x\|_0$ is minimized over a polyhedral set and the $\ell_0$-norm regularizer implicitly emphasizes sparsity of the solution. Such a setting captures a range of problems in image processing and statistical learning. Given the nonconvex and discontinuous nature of this norm, convex regularizers are often employed as substitutes. Therefore, far less is known about directly solving the $\ell_0$-minimization problem. Inspired by [19], we consider resolving an equivalent formulation of the $\ell_0$-minimization problem as a mathematical program with complementarity constraints (MPCC) and make the following contributions towards the characterization and computation of its KKT points: (i) First, we show that feasible points of this formulation satisfy the relatively weak Guignard constraint qualification. Furthermore, under the suitable convexity assumption on $f(x)$, an equivalence is derived between first-order KKT points and local minimizers of the MPCC formulation. (ii) Next, we apply two alternating direction method of multiplier (ADMM) algorithms to exploit special structure of the MPCC formulation: (ADMM$_{\rm cf}^{\mu, \alpha, \rho}$) and (ADMM$_{\rm cf}$). These two ADMM schemes both have tractable subproblems. Specifically, in spite of the overall nonconvexity, we show that the first of the ADMM updates can be effectively reduced to a closed-form expression by recognizing a hidden convexity property while the second necessitates solving a convex program. In (ADMM$_{\rm cf}^{\mu, \alpha, \rho}$), we prove subsequential convergence to a perturbed KKT point under mild assumptions. Our preliminary numerical experiments suggest that the tractable ADMM schemes are more scalable than their standard counterpart and ADMM$_{\rm cf}$ compares well with its competitors to solve the $\ell_0$-minimization problem., Comment: 47 pages, 3 tables

Stochastic dual dynamic programming is a cutting plane type algorithm for multi-stage stochastic optimization originated about 30 years ago. In spite of its popularity in practice, there does not exist any analysis on the convergence rates of this method. In this paper, we first establish the number of iterations, i.e., iteration complexity, required by a basic dual dynamic programming method for solving single-scenario multi-stage optimization problems, by introducing novel mathematical tools including the saturation of search points. We then refine these basic tools and establish the iteration complexity for an explorative dual dynamic programing method proposed herein and the classic stochastic dual dynamic programming method for solving more general multi-stage stochastic optimization problems under the standard stage-wise independence assumption. Our results indicate that the complexity of these methods mildly increases with the number of stages T, in fact linearly dependent on T for discounted problems. Therefore, they are efficient for strategic decision making which involves a large number of stages, but with a relatively small number of decision variables in each stage. Without explicitly discretizing the state and action spaces, these methods might also be pertinent to the related reinforcement learning and stochastic control areas.

This paper presents new Riemannian conjugate gradient methods and global convergence analyses under the strong Wolfe conditions. The main idea of the new methods is to combine the good global convergence properties of the Dai-Yuan method with the efficient numerical performance of the Hestenes-Stiefel method. The proposed methods compare well numerically with the existing methods for the Rayleigh quotient minimization problem on the unit sphere. Numerical comparisons show that they perform better than the existing ones., Comment: 17 pages, 2 figures and 2 tables

A key ingredient in branch and bound (B&B) solvers for mixed-integer programming (MIP) is the selection of branching variables since poor or arbitrary selection can affect the size of the resulting search trees by orders of magnitude. A recent article by Le Bodic and Nemhauser [Mathematical Programming, (2017)] investigated variable selection rules by developing a theoretical model of B&B trees from which they developed some new, effective scoring functions for MIP solvers. In their work, Le Bodic and Nemhauser left several open theoretical problems, solutions to which could guide the future design of variable selection rules. In this article, we first solve many of these open theoretical problems. We then implement an improved version of the model-based branching rules in SCIP 6.0, a state-of-the-art academic MIP solver, in which we observe an 11% geometric average time and node reduction on instances of the MIPLIB 2017 Benchmark Set that require large B&B trees.

Demiclosedness principles are powerful tools in the study of convergence of iterative methods. For instance, a multi-operator demiclosedness principle for firmly nonexpansive mappings is useful in obtaining simple and transparent arguments for the weak convergence of the shadow sequence generated by the Douglas–Rachford algorithm. We provide extensions of this principle, which are compatible with the framework of more general families of mappings such as cocoercive and conically averaged mappings. As an application, we derive the weak convergence of the shadow sequence generated by the adaptive Douglas–Rachford algorithm.

We study a family of optimal control problems under a set of controlled-loss constraints holding at different deterministic dates. The characterization of the associated value function by a Hamilton-Jacobi-Bellman equation usually calls for additional strong assumptions on the dynamics of the processes involved and the set of constraints. To treat this problem in absence of those assumptions, we first convert it into a state-constrained stochastic target problem and then apply a level-set approach. With this approach, the state constraints can be managed through an exact penalization technique. Accepted version Authors are grateful to the Young Investigator Training Program and Association of Bank Foundations. This is a post-peer-review, pre-copyedit version of an article published in Journal of Optimization Theory and Applications. The final authenticated version is available online at: http://dx.doi.org/10.1007/s10957-020-01724-8

The ancient concept of circumcenter has recently given birth to the Circumcentered-Reflection method (CRM). CRM was first employed to solve best approximation problems involving affine subspaces. In this setting, it was shown to outperform the most prestigious projection based schemes, namely, the Douglas-Rachford method (DRM) and the method of alternating projections (MAP). We now prove convergence of CRM for finding a point in the intersection of a finite number of closed convex sets. This striking result is derived under a suitable product space reformulation in which a point in the intersection of a closed convex set with an affine subspace is sought. It turns out that CRM skillfully tackles the reformulated problem. We also show that for a point in the affine set the CRM iteration is always closer to the solution set than both the MAP and DRM iterations. Our theoretical results and numerical experiments, showing outstanding performance, establish CRM as a powerful tool for solving general convex feasibility problems.

This paper develops new extremal principles of variational analysis that are motivated by applications to constrained problems of stochastic programming and semi-infinite programming without smoothness and/or convexity assumptions. These extremal principles concern measurable set-valued mappings/multifunctions with values in finite-dimensional spaces and are established in both approximate and exact forms. The obtained principles are instrumental to derive via variational approaches integral representations and upper estimates of regular and limiting normals cones to essential intersections of sets defined by measurable multifunctions, which are in turn crucial for novel applications to stochastic and semi-infinite programming., Comment: 26 pages

Motivated by a growing list of nontraditional statistical estimation problems of the piecewise kind, this paper provides a survey of known results supplemented with new results for the class of piecewise linear-quadratic programs. These are linearly constrained optimization problems with piecewise linear-quadratic (PLQ) objective functions. Starting from a study of the representation of such a function in terms of a family of elementary functions consisting of squared affine functions, squared plus-composite-affine functions, and affine functions themselves, we summarize some local properties of a PLQ function in terms of their first and second-order directional derivatives. We extend some well-known necessary and sufficient second-order conditions for local optimality of a quadratic program to a PLQ program and provide a dozen such equivalent conditions for strong, strict, and isolated local optimality, showing in particular that a PLQ program has the same characterizations for local minimality as a standard quadratic program. As a consequence of one such condition, we show that the number of strong, strict, or isolated local minima of a PLQ program is finite; this result supplements a recent result about the finite number of directional stationary objective values. Interestingly, these finiteness results can be uncovered by invoking a very powerful property of subanalytic functions; our proof is fairly elementary, however. We discuss applications of PLQ programs in some modern statistical estimation problems. These problems lead to a special class of unconstrained composite programs involving the non-differentiable $\ell_1$-function, for which we show that the task of verifying the second-order stationary condition can be converted to the problem of checking the copositivity of certain Schur complement on the nonnegative orthant.

The problem of recovering acoustic sources, more specifically monopoles, from point-wise measurements of the corresponding acoustic pressure at a limited number of frequencies is addressed. To this purpose, a family of sparse optimization problems in measure space in combination with the Helmholtz equation on a bounded domain is considered. A weighted norm with unbounded weight near the observation points is incorporated into the formulation. Optimality conditions and conditions for recovery in the small noise case are discussed, which motivates concrete choices of the weight. The numerical realization is based on an accelerated conditional gradient method in measure space and a finite element discretization.

Level-set methods for convex optimization are predicated on the idea that certain problems can be parameterized so that their solutions can be recovered as the limiting process of a root-finding procedure. This idea emerges time and again across a range of algorithms for convex problems. Here we demonstrate that strong duality is a necessary condition for the level-set approach to succeed. In the absence of strong duality, the level-set method identifies $$\epsilon $$ ϵ -infeasible points that do not converge to a feasible point as $$\epsilon $$ ϵ tends to zero. The level-set approach is also used as a proof technique for establishing sufficient conditions for strong duality that are different from Slater’s constraint qualification.

Given a set of sources and a set of sinks as points in the Euclidean plane, a directed network is a directed graph drawn in the plane with a directed path from each source to each sink. Such a network may contain nodes other than the given sources and sinks, called Steiner points. We characterize the local structure of the Steiner points in all shortest-length directed networks in the Euclidean plane. This characterization implies that these networks are constructible by straightedge and compass. Our results build on unpublished work of Alfaro, Campbell, Sher, and Soto from 1989 and 1990. Part of the proof is based on a new method that uses other norms in the plane. This approach gives more conceptual proofs of some of their results, and as a consequence, we also obtain results on shortest directed networks for these norms., 21 pages, 14 figures. To appear in Graphs and Combinatorics

We study a variant of the competitive facility location problem, in which a company is to locate new facilities in a market where competitor's facilities already exist. We consider the scenario where only a limited number of possible attractiveness levels is available, and the company has to select exactly one level for each open facility. The goal is to decide the facilities' locations and attractiveness levels that maximize the profit. We apply the gravity-based rule to model the behavior of the customers and formulate a multi-ratio linear fractional 0-1 program. Our main contributions are the exact solution approaches for the problem. These approaches allow for easy implementations without the need for designing complicated algorithms and are "friendly" to the users without a solid mathematical background. We conduct computational experiments on the randomly generated datasets to assess their computational performance. The results suggest that the mixed-integer quadratic conic approach outperforms the others in terms of computational time. Besides that, it is also the most straightforward one that only requires the users to be familiar with the general form of a conic quadratic inequality. Therefore, we recommend it as the primary choice for such a problem.

In this article, a globally convergent sequential quadratic programming (SQP) method is developed for multi-objective optimization problems with inequality type constraints. A feasible descent direction is obtained using a linear approximation of all objective functions as well as constraint functions. The sub-problem at every iteration of the sequence has feasible solution. A non-differentiable penalty function is used to deal with constraint violations. A descent sequence is generated which converges to a critical point under the Mangasarian-Fromovitz constraint qualification along with some other mild assumptions. The method is compared with a selection of existing methods on a suitable set of test problems., 19 pages, 11 figures

Originally developed in 1954, positive bases and positive spanning sets have been found to be a valuable concept in derivative-free optimization (DFO). The quality of a positive basis (or positive spanning set) can be quantified via the {\em cosine measure} and convergence properties of certain DFO algorithms are intimately linked to the value of this measure. However, it is unclear how to compute the cosine measure for a positive basis from the definition. In this paper, a deterministic algorithm to compute the cosine measure of any positive basis or finite positive spanning set is provided. The algorithm is proven to return the exact value of the cosine measure in finite time.

This work advances knowledge of the threshold of prox-boundedness of a function; an important concern in the use of proximal point optimization algorithms and in determining the existence of the Moreau envelope of the function. In finite dimensions, we study general prox-bounded functions and then focus on some useful families such as piecewise functions and Lipschitz continuous functions. The thresholds are explicitly determined when possible and bounds are established otherwise. Some calculus rules are constructed; we consider functions with known thresholds and find the thresholds of their sum and composition., 15 pages including references, 1 figure