Search

We propose a complete-search algorithm for solving a class of non-convex, possibly infinite-dimensional, optimization problems to global optimality. We assume that the optimization variables are in a bounded subset of a Hilbert space, and we determine worst-case run-time bounds for the algorithm under certain regularity conditions of the cost functional and the constraint set. Because these run-time bounds are independent of the number of optimization variables and, in particular, are valid for optimization problems with infinitely many optimization variables, we prove that the algorithm converges to an \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varepsilon $$\end{document}ε-suboptimal global solution within finite run-time for any given termination tolerance \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varepsilon > 0$$\end{document}ε>0. Finally, we illustrate these results for a problem of calculus of variations.

This paper formally introduces and studies a non-cooperative multi-agent game under uncertainty. The well-known Nash equilibrium is employed as the solution concept of the game. While there are several formulations of a stochastic Nash equilibrium problem, we focus mainly on a two-stage setting of the game wherein each agent is risk-averse and solves a rival-parameterized stochastic program with quadratic recourse. In such a game, each agent takes deterministic actions in the first stage and recourse decisions in the second stage after the uncertainty is realized. Each agent's overall objective consists of a deterministic first-stage component plus a second-stage mean-risk component defined by a coherent risk measure describing the agent's risk aversion. We direct our analysis towards a broad class of quantile-based risk measures and linear-quadratic recourse functions. For this class of non-cooperative games under uncertainty, the agents' objective functions can be shown to be convex in their own decision variables, provided that the deterministic component of these functions have the same convexity property. Nevertheless, due to the non-differentiability of the recourse functions, the agents' objective functions are at best directionally differentiable. Such non-differentiability creates multiple challenges for the analysis and solution of the game, two principal ones being: (1) a stochastic multi-valued variational inequality is needed to characterize a Nash equilibrium, provided that the players' optimization problems are convex; (2) one needs to be careful in the design of algorithms that require differentiability of the objectives. Moreover, the resulting (multi-valued) variational formulation cannot be expected to be of the monotone type in general. The main contributions of this paper are as follows: (a) Prior to addressing the main problem of the paper, we summarize several approaches that have existed in the literature to deal with uncertainty in a non-cooperative game. (b) We introduce a unified formulation of the two-stage SNEP with risk-averse players and convex quadratic recourse functions and highlight the technical challenges in dealing with this game. (c) To handle the lack of smoothness, we propose smoothing schemes and regularization that lead to differentiable approximations. (d) To deal with non-monotonicity, we impose a generalized diagonal dominance condition on the players' smoothed objective functions that facilitates the application and ensures the convergence of an iterative best-response scheme. (e) To handle the expectation operator, we rely on known methods in stochastic programming that include sampling and approximation. (f) We provide convergence results for various versions of the best-response scheme, particularly for the case of private recourse functions. Overall, this paper lays the foundation for future research into the class of SNEPs that provides a constructive paradigm for modeling and solving competitive decision making problems with risk-averse players facing uncertainty; this paradigm is very much at an infancy stage of research and requires extensive treatment in order to meet its broad applications in many engineering and economics domains.

In this paper we continue the studies of the persistent effect of attraction of Newton-type iterations for optimality systems to critical Lagrange multipliers. It appears very important to understand the nature of this striking phenomenon, in particular, because it is precisely the reason of slow convergence of such methods when applied to problems with degenerate constraints. All previously known results concerned with this effect were a posteriori by nature: they were showing that in case of convergence, the dual limit is in a sense unlikely to be noncritical. This paper suggests the first a priori result in this direction, showing that critical multipliers actually serve as attractors: for a fully quadratic optimization problem with equality constraints, under certain reasonable assumptions we establish actual local convergence of the Newton---Lagrange method to a critical multiplier starting from a "dense" set around a given critical multiplier. This is an important step forward in understanding the attraction phenomenon.

This paper presents a methodology for using varying sample sizes in batch-type optimization methods for large-scale machine learning problems. The first part of the paper deals with the delicate issue of dynamic sample selection in the evaluation of the function and gradient. We propose a criterion for increasing the sample size based on variance estimates obtained during the computation of a batch gradient. We establish an $${O(1/\epsilon)}$$ complexity bound on the total cost of a gradient method. The second part of the paper describes a practical Newton method that uses a smaller sample to compute Hessian vector-products than to evaluate the function and the gradient, and that also employs a dynamic sampling technique. The focus of the paper shifts in the third part of the paper to L 1-regularized problems designed to produce sparse solutions. We propose a Newton-like method that consists of two phases: a (minimalistic) gradient projection phase that identifies zero variables, and subspace phase that applies a subsampled Hessian Newton iteration in the free variables. Numerical tests on speech recognition problems illustrate the performance of the algorithms.

This paper studies the local analysis of equations on a product U × U of Banach spaces, whose variables lie in a subset having the special property that it is locally Lipschitz-homeomorphic to an open subset of U. A prominent example, to which we devote most of the paper, is a system of equations whose variables lie in the graph of a maximal monotone operator. This general formulation covers many specific problems of interest, and our objective is to understand the local behavior of solutions of such equations when they depend on parameters. We analyze this local behavior in stages, the first stage being to apply a tailored implicit-function theorem to an abstract formulation of the problem, thereby producing a set of results applicable to any particular problem instance. The second stage is to specialize the analysis to a Hilbert space H, with the subset mentioned above being the graph of a maximal monotone operator on H. This makes the results of the first stage applicable to many variational problems of practical importance. We then develop in detail the analytical steps to apply these results to finite-dimensional variational conditions with constraints of generalized nonlinear-programming type. The conditions thus identified generalize the strong second-order sufficient condition and linear-independence constraint qualification of nonlinear programming. A detailed example brings out some of the issues involved in practical implementation of this method. It also shows that aspects of representation (problem formulation) can strongly influence the feasibility of local analysis. This sensitivity to representation does not seem to be well known.

In this paper, we provide theoretical analysis for a cubic regularization of Newton method as applied to unconstrained minimization problem. For this scheme, we prove general local convergence results. However, the main contribution of the paper is related to global worst-case complexity bounds for different problem classes including some nonconvex cases. It is shown that the search direction can be computed by standard linear algebra technique.

We study a multi-period oligopolistic market for a single perishable product with fixed inventory. Our goal is to address the competitive aspect of the problem together with demand uncertainty using ideas from robust optimization and variational inequalities. The demand function for each seller has some associated uncertainty and we assume that the sellers would like to adopt a policy that is robust to adverse uncertain circumstances. We believe this is the first paper that uses robust optimization for dynamic pricing under competition. In particular, starting with a given fixed inventory, each seller competes over a multi-period time horizon in the market by setting prices and protection levels for each period at the beginning of the time horizon. Any unsold inventory at the end of the horizon is worthless. The sellers do not have the option of periodically reviewing and replenishing their inventory. We study non-cooperative Nash equilibrium policies for sellers under such a model. This kind of a setup can be used to model pricing of air fares, hotel reservations, bandwidth in communication networks, etc. In this paper we demonstrate our results through some numerical examples.

This paper is concerned with characterizations of nonsmooth saddle critical points for numerical algorithm design. Most characterizations for nonsmooth saddle critical points in the literature focus on existence issue and are converted to solve global minimax problems. Thus they are not helpful for numerical algorithm design. Inspired by the results on computational theory and methods for finding multiple smooth saddle critical points in [14, 15, 19, 21, 23], a local minimax characterization for multiple nonsmooth saddle critical points in either a Hilbert space or a reflexive Banach space is established in this paper to provide a mathematical justification for numerical algorithm design. A local minimax algorithm for computing multiple nonsmooth saddle critical points is presented by its flow chart.

This paper is devoted to the numerical resolution of unit-commitment problems, with emphasis on the French model optimizing the daily production of electricity. The solution process has two phases. First a Lagrangian relaxation solves the dual to find a lower bound; it also gives a primal relaxed solution. We then propose to use the latter in the second phase, for a heuristic resolution based on a primal proximal algorithm. This second step comes as an alternative to an earlier approach, based on augmented Lagrangian (i.e. a dual proximal algorithm). We illustrate the method with some real-life numerical results. A companion paper is devoted to a theoretical study of the heuristic in the second phase.

The authors of this paper recently introduced a transformation [4] that converts a class of semidefinite programs (SDPs) into nonlinear optimization problems free of matrix-valued constraints and variables. This transformation enables the application of nonlinear optimization techniques to the solution of certain SDPs that are too large for conventional interior-point methods to handle efficiently. Based on the transformation, we proposed a globally convergent, first-order (i.e., gradient-based) log-barrier algorithm for solving a class of linear SDPs. In this paper, we discuss an efficient implementation of the proposed algorithm and report computational results on semidefinite relaxations of three types of combinatorial optimization problems. Our results demonstrate that the proposed algorithm is indeed capable of solving large-scale SDPs and is particularly effective for problems with a large number of constraints.

In this paper, we present a new framework of bi-level unconstrained minimization for development of accelerated methods in Convex Programming. These methods use approximations of the high-order proximal points, which are solutions of some auxiliary parametric optimization problems. For computing these points, we can use different methods, and, in particular, the lower-order schemes. This opens a possibility for the latter methods to overpass traditional limits of the Complexity Theory. As an example, we obtain a new second-order method with the convergence rate $$O\left( k^{-4}\right) $$ O k - 4 , where k is the iteration counter. This rate is better than the maximal possible rate of convergence for this type of methods, as applied to functions with Lipschitz continuous Hessian. We also present new methods with the exact auxiliary search procedure, which have the rate of convergence $$O\left( k^{-(3p+1)/ 2}\right) $$ O k - ( 3 p + 1 ) / 2 , where $$p \ge 1$$ p ≥ 1 is the order of the proximal operator. The auxiliary problem at each iteration of these schemes is convex.

The notion of best approximation mapping (BAM) with respect to a closed affine subspace in finite-dimensional space was introduced by Behling, Bello Cruz and Santos to show the linear convergence of the block-wise circumcentered-reflection method. The best approximation mapping possesses two critical properties of the circumcenter mapping for linear convergence. Because the iteration sequence of BAM linearly converges, the BAM is interesting in its own right. In this paper, we naturally extend the definition of BAM from closed affine subspace to nonempty closed convex set and from $\mathbb{R}^{n}$ to general Hilbert space. We discover that the convex set associated with the BAM must be the fixed point set of the BAM. Hence, the iteration sequence generated by a BAM linearly converges to the nearest fixed point of the BAM. Connections between BAMs and other mappings generating convergent iteration sequences are considered. Behling et al.\ proved that the finite composition of BAMs associated with closed affine subspaces is still a BAM in $\mathbb{R}^{n}$. We generalize their result from $\mathbb{R}^{n}$ to general Hilbert space and also construct a new constant associated with the composition of BAMs. This provides a new proof of the linear convergence of the method of alternating projections. Moreover, compositions of BAMs associated with general convex sets are investigated. In addition, we show that convex combinations of BAMs associated with affine subspaces are BAMs. Last but not least, we connect BAM with circumcenter mapping in Hilbert spaces., 31 pages and 2 figures

We study the properties of the weighted central paths in linear programming. We consider each path as the function of the parameter 0 where the value at D 0 corresponds to the limit point at the boundary of the feasible set. We calculate the recursive formulas for the central path derivatives of all orders valid at each 0. We establish the geometric growth of the derivatives and, consequently, the analyticity of the weighted central path at 0 . This paper also provides the analysis of limiting behavior of those projection operators which often appear in the interior point methods. The central trajectory is one of the most important concepts in the interior point methods for linear programming. The central trajectory (or the central path) is a distinguished curve that is interior to the feasible set and tends to an optimal point. Most interior points algorithms follow, or are related to, the central trajectory. Although the central path concept is essential for the design and analysis of algorithms relatively few papers study properties of this curve itself. For a review on this topic we refer to (3). Recently Zhao and Zhu (16) have studied some analytical properties of the central trajectory in the interior of the feasible set. They have established an upper bound for the high order derivatives of the weighted central path and estimated the convergence radius of the corresponding Taylor series. They have also pointed out the relationship of these properties with the complexity of a path following algorithm where the number of iterations is estimated by a "curvature integral". (See also (15).) Further, Guler (3) has analysed some limiting properties of the central trajectory derivatives with respect to the parameter >0. He has established the existence of finite limits for derivatives of all orders as tends to infinity (that is, the central path stays in the interior) and to zero (the central path tends to the boundary). These results have applications to the construction of polynomial- time algorithms which are also locally fast. A surprising result of this study is that all derivatives of order > 1t end to 0 as!1, which helps to explain why interior point methods for linear programming require so few iterations (see (3)).

This paper introduces tensor methods for solving large sparse systems of nonlinear equations. Tensor methods for nonlinear equations were developed in the context of solving small to medium-sized dense problems. They base each iteration on a quadratic model of the nonlinear equations, where the second-order term is selected so that the model requires no more derivative or function information per iteration than standard linear model-based methods, and hardly more storage or arithmetic operations per iteration. Computational experiments on small to medium-sized problems have shown tensor methods to be considerably more efficient than standard Newton-based methods, with a particularly large advantage on singular problems. This paper considers the extension of this approach to solve large sparse problems. The key issue considered is how to make efficient use of sparsity in forming and solving the tensor model problem at each iteration. Accomplishing this turns out to require an entirely new way of solving the tensor model that successfully exploits the sparsity of the Jacobian, whether the Jacobian is nonsingular or singular. We develop such an approach and, based upon it, an efficient tensor method for solving large sparse systems of nonlinear equations. Test results indicate that this tensor method is significantly more efficient and robust than an efficient sparse Newton-based method, in terms of iterations, function evaluations, and execution time. © 1998 The Mathematical Programming Society, Inc. Published by Elsevier Science B.V.

The subject of this paper is the systematic study of second order notions concerning differentiable functions with Lipschitz derivative. The results and notions are motivated by recent papers of Cominetti, Correa and Hiriart-Urruty. The first goal of this paper is the comparison of several known second order directional derivatives. The second goal is the introduction of a generalized Hessian which is a set of certain symmetric bilinear forms. The relation of this generalized Hessian to other existing second order derivatives is also described.

In this paper we describe a cutting plane algorithm for the Steiner tree packing problem. We use our algorithm to solve some switchbox routing problems of VLSI-design and report on our computational experience. This includes a brief discussion of separation algorithms, a new LP-based primal heuristic and implementation details. The paper is based on the polyhedral theory for the Steiner tree packing polyhedron developed in our companion paper (this issue) and meant to turn this theory into an algorithmic tool for the solution of practical problems.

LetG=(V, E) be a graph andT⊆V be a node set. We call an edge setS a Steiner tree forT ifS connects all pairs of nodes inT. In this paper we address the following problem, which we call the weighted Steiner tree packing problem. Given a graphG=(V, E) with edge weightsw e , edge capacitiesc e ,e∈E, and node setT 1,…,T N , find edge setsS 1,…,S N such that eachS k is a Steiner tree forT k , at mostc e of these edge sets use edgee for eache∈E, and the sum of the weights of the edge sets is minimal. Our motivation for studying this problem arises from a routing problem in VLSI-design, where given sets of points have to be connected by wires. We consider the Steiner tree packing problem from a polyhedral point of view and define an associated polyhedron, called the Steiner tree packing polyhedron. The goal of this paper is to (partially) describe this polyhedron by means of inequalities. It turns out that, under mild assumptions, each inequality that defines a facet for the (single) Steiner tree polyhedron can be lifted to a facet-defining inequality for the Steiner tree packing polyhedron. The main emphasis of this paper lies on the presentation of so-called joint inequalities that are valid and facet-defining for this polyhedron. Inequalities of this kind involve at least two Steiner trees. The classes of inequalities we have found form the basis of a branch & cut algorithm. This algorithm is described in our companion paper (in this issue).

This is the second part of two papers addressing the study of the facial structure of the Steiner tree polyhedron. In this paper we identify several classes of facet defining inequalities and relate them to special classes of graphs on which the Steiner tree problem is known to be NP-hard.

This paper presents a modified damped Newton algorithm for solving variational inequality problems based on formulating this problem as a system of equations using the Minty map. The proposed modified damped-Newton method insures convergence and locally quadratic convergence under the assumption of regularity. Under the assumption ofweak regularity and some mild conditions, the modified algorithm is shown to always create a descent direction and converge to the solution. Hence, this new algorithm is often suitable for many applications where regularity does not hold. Part II of this paper presents the results of extensive computational testing of this new method.

The graphical relaxation of the Traveling Salesman Problem is the relaxation obtained by requiring that the salesman visit each city at least once instead of exactly once. This relaxation has already led to a better understanding of the Traveling Salesman polytope in Cornuejols, Fonlupt and Naddef (1985). We show here how one can compose facet-inducing inequalities for the graphical traveling salesman polyhedron, and obtain other facet-inducing inequalities. This leads to new valid inequalities for the Symmetric Traveling Salesman polytope. This paper is the first of a series of three papers on the Symmetric Traveling Salesman polytope, the next one studies the strong relationship between that polytope and its graphical relaxation, and the last one applies all the theoretical developments of the two first papers to prove some new facet-inducing results.

In this paper, we propose a semi-metric for Markov processes that allows to bound optimal values of linear Markovian stochastic optimization problems. Similar to existing notions of distance for general stochastic processes, our distance is based on transportation metrics. As opposed to the extant literature, the proposed distance is problem specific, i.e., dependent on the data of the problem whose objective value we want to bound. As a result, we are able to consider problems with randomness in the constraints as well as in the objective function and therefore relax an assumption in the extant literature. We derive several properties of the proposed semi-metric and demonstrate its use in a stylized numerical example.

This paper is devoted to the study of sensitivity to perturbation of parametrized variational inclusions involving maximally monotone operators in a Hilbert space. The perturbation of all the data involved in the problem is taken into account. Using the concept of proto-differentiability of a multifunction and the notion of semi-differentiability of a single-valued map, we establish the differentiability of the solution of a parametrized monotone inclusion. We also give an exact formula of the proto-derivative of the resolvent operator associated to the maximally monotone parameterized variational inclusion. This shows that the derivative of the solution of the parametrized variational inclusion obeys the same pattern by being itself a solution of a variational inclusion involving the semi-derivative and the proto-derivative of the associated maps. An application to the study of the sensitivity analysis of a parametrized primal-dual composite monotone inclusion is given. Under some sufficient conditions on the data, it is shown that the primal and the dual solutions are differentiable and their derivatives belong to the derivative of the associated Kuhn–Tucker set.

A distributionally robust joint chance constraint involves a set of uncertain linear inequalities which can be violated up to a given probability threshold $$\epsilon $$ , over a given family of probability distributions of the uncertain parameters. A conservative approximation of a joint chance constraint, often referred to as a Bonferroni approximation, uses the union bound to approximate the joint chance constraint by a system of single chance constraints, one for each original uncertain constraint, for a fixed choice of violation probabilities of the single chance constraints such that their sum does not exceed $$\epsilon $$ . It has been shown that, under various settings, a distributionally robust single chance constraint admits a deterministic convex reformulation. Thus the Bonferroni approximation approach can be used to build convex approximations of distributionally robust joint chance constraints. In this paper we consider an optimized version of Bonferroni approximation where the violation probabilities of the individual single chance constraints are design variables rather than fixed a priori. We show that such an optimized Bonferroni approximation of a distributionally robust joint chance constraint is exact when the uncertainties are separable across the individual inequalities, i.e., each uncertain constraint involves a different set of uncertain parameters and corresponding distribution families. Unfortunately, the optimized Bonferroni approximation leads to NP-hard optimization problems even in settings where the usual Bonferroni approximation is tractable. When the distribution family is specified by moments or by marginal distributions, we derive various sufficient conditions under which the optimized Bonferroni approximation is convex and tractable. We also show that for moment based distribution families and binary decision variables, the optimized Bonferroni approximation can be reformulated as a mixed integer second-order conic set. Finally, we demonstrate how our results can be used to derive a convex reformulation of a distributionally robust joint chance constraint with a specific non-separable distribution family.

The notion of perfect equilibrium was formulated by Selten (Int J Game Theory 4(1):25–55, 1975) as a strict refinement of Nash equilibrium. For an extensive-form game with perfect recall, every perfect equilibrium of its agent normal-form game yields a perfect equilibrium of the extensive-form game. This paper aims to develop a differentiable homotopy method for computing perfect equilibria of normal-form games. To accomplish this objective, we constitute an artificial game by introducing a continuously differentiable function of an extra variable. The artificial game defines a differentiable homotopy mapping and establishes the existence of a smooth path to a perfect equilibrium. For numerical comparison, we also describe a simplicial homotopy method. Numerical results show that the differentiable homotopy method is numerically stable and efficient and significantly outperforms the simplicial homotopy method especially when the problem is large.

The paper conducts a second-order variational analysis for an important class of nonpolyhedral conic programs generated by the so-called second-order/Lorentz/ice-cream cone \({\mathcal {Q}}\). From one hand, we prove that the indicator function of \({\mathcal {Q}}\) is always twice epi-differentiable and apply this result to characterizing the uniqueness of Lagrange multipliers together with an error bound estimate in the general second-order cone programming setting involving twice differentiable data. On the other hand, we precisely calculate the graphical derivative of the normal cone mapping to \({\mathcal {Q}}\) under the metric subregularity constraint qualification and then give an application of the latter result to a complete characterization of isolated calmness for perturbed variational systems associated with second-order cone programs. The obtained results seem to be the first in the literature in these directions for nonpolyhedral problems without imposing any nondegeneracy assumptions.

This paper proposes tight semidefinite relaxations for polynomial optimization. The optimality conditions are investigated. We show that generally Lagrange multipliers can be expressed as polynomial functions in decision variables over the set of critical points. The polynomial expressions can be determined by linear equations. Based on these expressions, new Lasserre type semidefinite relaxations are constructed for solving polynomial optimization. We show that the hierarchy of new relaxations has finite convergence, or equivalently, the new relaxations are tight for a finite relaxation order., Comment: 27 pages

Given a closed convex cone C in a finite dimensional real Hilbert space H, a weakly homogeneous map $$f\,{:}\,C\rightarrow H$$ is a sum of two continuous maps h and g, where h is positively homogeneous of degree $$\gamma $$ ($$\ge 0$$) on C and $$g(x)=o(||x||^\gamma )$$ as $$||x||\rightarrow \infty $$ in C. Given such a map f, a nonempty closed convex subset K of C, and a $$q\in H$$, we consider the variational inequality problem, $${\text {VI}}(f,K,q)$$, of finding an $$x^*\in K$$ such that $$\langle f(x^*)+q,x-x^*\rangle \ge 0$$ for all $$x\in K.$$ In this paper, we establish some results connecting the variational inequality problem $${\text {VI}}(f,K,q)$$ and the cone complementarity problem $${\text {CP}}(f^{\infty },K^{\infty },0)$$, where $$f^{\infty }:=h$$ is the homogeneous part of f and $$K^{\infty }$$ is the recession cone of K. We show, for example, that $${\text {VI}}(f,K,q)$$ has a nonempty compact solution set for every q when zero is the only solution of $${\text {CP}}(f^{\infty },K^{\infty },0)$$ and the (topological) index of the map $$x\mapsto x-\Pi _{K^{\infty }}(x-G(x))$$ at the origin is nonzero, where G is a continuous extension of $$f^{\infty }$$ to H. As a consequence, we generalize a complementarity result of Karamardian (J Optim Theory Appl 19:227–232, 1976) formulated for homogeneous maps on proper cones to variational inequalities. The results above extend some similar results proved for affine variational inequalities and for polynomial complementarity problems over the nonnegative orthant in $${\mathcal {R}}^n$$. As an application, we discuss the solvability of nonlinear equations corresponding to weakly homogeneous maps over closed convex cones. In particular, we extend a result of Hillar and Johnson (Proc Am Math Soc 132:945–953, 2004) on the solvability of symmetric word equations to Euclidean Jordan algebras.

Multistage stochastic integer programming (MSIP) combines the difficulty of uncertainty, dynamics, and non-convexity, and constitutes a class of extremely challenging problems. A common formulation for these problems is a dynamic programming formulation involving nested cost-to-go functions. In the linear setting, the cost-to-go functions are convex polyhedral, and decomposition algorithms, such as nested Benders’ decomposition and its stochastic variant, stochastic dual dynamic programming (SDDP), which proceed by iteratively approximating these functions by cuts or linear inequalities, have been established as effective approaches. However, it is difficult to directly adapt these algorithms to MSIP due to the nonconvexity of integer programming value functions. In this paper we propose an extension to SDDP—called stochastic dual dynamic integer programming (SDDiP)—for solving MSIP problems with binary state variables. The crucial component of the algorithm is a new reformulation of the subproblems in each stage and a new class of cuts, termed Lagrangian cuts, derived from a Lagrangian relaxation of a specific reformulation of the subproblems in each stage, where local copies of state variables are introduced. We show that the Lagrangian cuts satisfy a tightness condition and provide a rigorous proof of the finite convergence of SDDiP with probability one. We show that, under fairly reasonable assumptions, an MSIP problem with general state variables can be approximated by one with binary state variables to desired precision with only a modest increase in problem size. Thus our proposed SDDiP approach is applicable to very general classes of MSIP problems. Extensive computational experiments on three classes of real-world problems, namely electric generation expansion, financial portfolio management, and network revenue management, show that the proposed methodology is very effective in solving large-scale multistage stochastic integer optimization problems.

Two players, the Defender and the Attacker play the following game. A matroid $$M=(S,\mathcal {I})$$M=(S,I), a weight function $$d:S\rightarrow \mathbb {R}^+$$d:SźR+ and a cost function $$c:S\rightarrow \mathbb {R}$$c:SźR are given. The Defender chooses a base B of the matroid M and the Attacker chooses an element $$s\in S$$sźS of the ground set. In all cases, the Attacker pays the Defender the cost of attack c(s). Besides that, if $$s\in B$$sźB then the Defender pays the Attacker the amount d(s); if, on the other hand, $$s\notin B$$sźB then there is no further payoff. Special cases of this two-player, zero-sum game were considered and solved in various security-related applications. In this paper we show that it is also possible to compute Nash-equilibrium mixed strategies for both players in strongly polynomial time in the above general matroid setting. We also consider a further generalization where common bases of two matroids are chosen by the Defender and apply this to define and efficiently compute a new reliability metric on digraphs.

The paper presents CONOPT, an optimization system for static and dynamic large scale nonlinearly constrained optimization problems. The system is based on the GRG algorithm. All computations involving the Jacobian of the constraints use sparse matrix algorithm from linear programming modified to take optimal advantage of the nonlinearity. The paper presents the main features of the system, especially the inversion routines and their data structures, the dynamic setting of tolerances in Newton's algorithm, and the user features in the overall packaging. Computational experience with some medium to large models is presented.

This paper examines a type of symmetric quasi-Newton update for use in nonlinear optimization algorithms. The updates presented here impose additional properties on the Hessian approximations that do not result if the usual quasi-Newton updating schemes are applied to certain Gibbs free energy minimization problems. The updates derived in this paper are symmetric matrices that satisfy a given matrix equation and are least squares solutions to the secant equation. A general representation for this class of updates is given. The update in this class that has the minimum weighted Frobenius norm is also presented.

This and a companion paper consider how current implementations of the simplex method may be adapted to better solve linear programs that have a staged, or ‘staircase’, structure. The present paper looks at ‘inversion’ routines within the simplex method, particularly those for sparse triangular factorization of a basis by Gaussian elimination and for solution of triangular linear systems. The succeeding paper examines ‘pricing’ routines. Both papers describe extensive (though preliminary) computational experience, and can point to some quite promising results.

Jack Edmonds developed a new way of looking at extremal combinatorial problems and applied his technique with a great success to the problems of the maximal-weight degreeconstrained subgraphs. Professor C. St. J.A. Nash-Williams suggested to use Edmonds' approach in the context of hamiltonian graphs. In the present paper, we determine a new set of inequalities (the “comb inequalities”) which are satisfied by the characteristic functions of hamiltonian circuits but are not explicit in the straightforward integer programming formulation. A direct application of the linear programming duality theorem then leads to a new necessary condition for the existence of hamiltonian circuits; this condition appears to be stronger than the ones previously known. Relating linear programming to hamiltonian circuits, the present paper can also be seen as a continuation of the work of Dantzig, Fulkerson and Johnson on the traveling salesman problem.

This paper addresses the problem of selecting the parameter in a family of algorithms for unconstrained minimization known as Self Scaling Variable Metric (SSVM) Algorithms. This family, that has some very attractive properties, is based on a two parameter formula for updating the inverse Hessian approximation, in which the parameters can take any values between zero and one. Earlier results obtained for SSVM algorithms apply to the entire family and give no indication of how the choice of parameter may affect the algorithm's performance. In this paper, we examine empirically the effect of varying the parameters and relaxing the line-search. Theoretical consideration also leads to a switching tule for these parameters. Numerical results obtained for the SSVM algorithm indicate that with proper parameter selection it is superior to the DFP algorithm, particularly for high-dimensional problems.

This paper presents an iterative, Newton-type method for solving a class of linear complementarity problems. This class was discovered by Mangasarian who had established that these problems can be solved as linear programs. Cottle and Pang characterized solutions of the problems in terms of least elements of certain polyhedral sets. The algorithms developed in this paper are shown to converge to the least element solutions. Some applications and computational results are also discussed.

In an earlier paper [6] it is shown how the use of symmetric additions of rows and columns enables a stableLDLT factorization of symmetric indefinite matrices. In this paper we describe partial pivoting strategies. These strategies are faster than the complete pivoting strategies that were introduced in the first paper. Numerical experiments are included. The results show that some of the new strategies share the stable behaviour of complete pivoting while running almost as fast as the well-known Cholesky decomposition.

In a series of recent papers, Oren, Oren and Luenberger, Oren and Spedicato, and Spedicato have developed the self-scaling variable metric algorithms. These algorithms alter Broyden's single parameter family of approximations to the inverse Hessian to a double parameter family. Conditions are given on the new parameter to minimize a bound on the condition number of the approximated inverse Hessian while insuring improved step-wise convergence. Davidon has devised an update which also minimizes the bound on the condition number while remaining in the Broyden single parameter family. This paper derives initial scalings for the approximate inverse Hessian which makes members of the Broyden class self-scaling. The Davidon, BFGS, and Oren—Spedicato updates are tested for computational efficiency and stability on numerous test functions, with the results indicating strong superiority computationally for the Davidon and BFGS update over the self-scaling update, except on a special class of functions, the homogeneous functions.

International audience; The general theory of Lyapunov stability of first-order differential inclu- sions in Hilbert spaces has been studied by the authors in the previous paper (Adly et al. in Nonlinear Anal 75(3): 985–1008, 2012). This new contribution focuses on the case when the interior of the domain of the maximally monotone operator governing the given differential inclusion is nonempty; this includes in a natural way the finite- dimensional case. The current setting leads to simplified, more explicit criteria and permits some flexibility in the choice of the generalized subdifferentials. Some con- sequences of the viability of closed sets are given. Our analysis makes use of standard tools from convex and variational analysis.

We study the Celis---Dennis---Tapia (CDT) problem: minimize a non-convex quadratic function over the intersection of two ellipsoids. In contrast to the well-studied trust region problem where the feasible set is just one ellipsoid, the CDT problem is not yet fully understood. Our main objective in this paper is to narrow the difficulty gap that occurs when the Hessian of the Lagrangian is indefinite at all Karush---Kuhn---Tucker points. We prove new sufficient and necessary conditions both for local and global optimality, based on copositivity, giving a complete characterization in the degenerate case.

In this paper, we consider the so-called worst-case linear optimization (WCLO) with uncertainties in the right-hand-side of the constraints. Such a problem arises from numerous applications such as systemic risk estimate in finance and stochastic optimization. We first show the problem is NP-hard and present a coarse semidefinite relaxation (SDR) for WCLO. An iterative procedure is introduced to sequentially refine the relaxation model based on the solution of the current relaxation model by simply changing some parameters in the coarse SDR. We show that the sequence of the proposed SDRs will converge to some nonlinear semidefinite optimization problem (SDO). A bi-section search algorithm is proposed to solve the resulting nonlinear SDO. Our preliminary experimental results illustrate that the nonlinear SDR can provide very tight bounds for the original WCLO and is able to locate the exact global solution in most cases within a few iterations on average.

Graph partitioning is an intractable problem that arises in many practical applications. Heuristics such as local search generate good (though suboptimal) solutions in limited time. Such heuristics must be able to explore the solution space quickly and, when the solution space is constrained, differentiate feasible solutions from infeasible ones. Geographic zoning problems allocate some resource (e.g., political representation, school enrollment, police patrols) to contiguous zones modeled by partitions of an embedded planar graph. Each vertex corresponds to an area of the plane (e.g., census block, town, county), and local search moves one area from its current zone to a different zone in each iteration. Enforcing contiguity constraints may require significant computation when the graph is large. While existing algorithms require linear or polylogarithmic time (in the number of vertices) to assess contiguity in each local search iteration, the geo-graph paradigm shows how contiguity can be verified by examining only the set of vertices that border the transferred area (i.e., those areas whose boundaries share at least a single point with the boundary of the transferred area). This paper develops efficient algorithms that examine these vertices more quickly than traditional search-based methods, allowing practitioners to more fully consider their zoning options when creating zones with local search.

In a two-stage robust covering problem, one of several possible scenarios will appear tomorrow and require to be covered, but costs are higher tomorrow than today. What should you anticipatorily buy today, so that the worst-case cost (summed over both days) is minimized? We consider the $$k$$ k -robust model where the possible scenarios tomorrow are given by all demand-subsets of size $$k$$ k . In this paper, we give the following simple and intuitive template for $$k$$ k -robust covering problems: having built some anticipatory solution, if there exists a single demand whose augmentation cost is larger than some threshold, augment the anticipatory solution to cover this demand as well, and repeat. We show that this template gives good approximation algorithms for $$k$$ k -robust versions of many standard covering problems: set cover, Steiner tree, Steiner forest, minimum-cut and multicut. Our $$k$$ k -robust approximation ratios nearly match the best bounds known for their deterministic counterparts. The main technical contribution lies in proving certain net-type properties for these covering problems, which are based on dual-rounding and primal---dual ideas; these properties might be of some independent interest. As a by-product of our techniques, we also get algorithms for max---min problems of the form: "given a covering problem instance, which $$k$$ k of the elements are costliest to cover?" For the problems mentioned above, we show that their $$k$$ k -max---min versions have performance guarantees similar to those for the $$k$$ k -robust problems.

Supply function equilibria are used in the analysis of divisible good auctions with a large number of identical objects to be sold or bought. An important example occurs in wholesale electricity markets. Despite the substantial literature on supply function equilibria the existence of a pure strategy Nash equilibria for a uniform price auction in asymmetric cases has not been established in a general setting. In this paper we prove the existence of a supply function equilibrium for a duopoly with asymmetric firms having convex non-decreasing marginal costs, with decreasing concave demand subject to an additive demand shock, provided that the second derivative of the demand function is small enough and not increasing. The proof is constructive and also gives insight into the structure of the equilibrium solutions.

This paper studies the $H^1$ Sobolev seminorm of quadratic functions. The research is motivated by the least-norm interpolation that is widely used in derivative-free optimization. We express the $H^1$ seminorm of a quadratic function explicitly in terms of the Hessian and the gradient when the underlying domain is a ball. The seminorm gives new insights into least-norm interpolation. It clarifies the analytical and geometrical meaning of the objective function in least-norm interpolation. We employ the seminorm to study the extended symmetric Broyden update proposed by Powell. Numerical results show that the new thoery helps improve the performance of the update. Apart from the theoretical results, we propose a new method of comparing derivative-free solvers, which is more convincing than merely counting the numbers of function evaluations., 17 pages, 6 figures

Consider the optimization problem of minimizing a polynomial function subject to polynomial constraints. A typical approach for solving it globally is applying Lasserre’s hierarchy of semidefinite relaxations, based on either Putinar’s or Schmudgen’s Positivstellensatz. A practical question in applications is: how to certify its convergence and get minimizers? In this paper, we propose flat truncation as a certificate for this purpose. Assume the set of global minimizers is nonempty and finite. Our main results are: (1) Putinar type Lasserre’s hierarchy has finite convergence if and only if flat truncation holds, under some generic assumptions; the same conclusion holds for the Schmudgen type one under weaker assumptions. (2) Flat truncation is asymptotically satisfied for Putinar type Lasserre’s hierarchy if the archimedean condition holds; the same conclusion holds for the Schmudgen type one if the feasible set is compact. (3) We show that flat truncation can be used as a certificate to check exactness of standard SOS relaxations and Jacobian SDP relaxations.

Interpolation-based trust-region methods are an important class of algorithms for Derivative-Free Optimization which rely on locally approximating an objective function by quadratic polynomial interpolation models, frequently built from less points than there are basis components. Often, in practical applications, the contribution of the problem variables to the objective function is such that many pairwise correlations between variables are negligible, implying, in the smooth case, a sparse structure in the Hessian matrix. To be able to exploit Hessian sparsity, existing optimization approaches require the knowledge of the sparsity structure. The goal of this paper is to develop and analyze a method where the sparse models are constructed automatically. The sparse recovery theory developed recently in the field of compressed sensing characterizes conditions under which a sparse vector can be accurately recovered from few random measurements. Such a recovery is achieved by minimizing the l 1-norm of a vector subject to the measurements constraints. We suggest an approach for building sparse quadratic polynomial interpolation models by minimizing the l 1-norm of the entries of the model Hessian subject to the interpolation conditions. We show that this procedure recovers accurate models when the function Hessian is sparse, using relatively few randomly selected sample points. Motivated by this result, we developed a practical interpolation-based trust-region method using deterministic sample sets and minimum l 1-norm quadratic models. Our computational results show that the new approach exhibits a promising numerical performance both in the general case and in the sparse one.

Affine generalized Nash equilibrium problems (AGNEPs) represent a class of non-cooperative games in which players solve convex quadratic programs with a set of (linear) constraints that couple the players’ variables. The generalized Nash equilibria (GNE) associated with such games are given by solutions to a linear complementarity problem (LCP). This paper treats a large subclass of AGNEPs wherein the coupled constraints are shared by, i.e., common to, the players. Specifically, we present several avenues for computing structurally different GNE based on varying consistency requirements on the Lagrange multipliers associated with the shared constraints. Traditionally, variational equilibria (VE) have been amongst the more well-studied GNE and are characterized by a requirement that the shared constraint multipliers be identical across players. We present and analyze a modification to Lemke’s method that allows us to compute GNE that are not necessarily VE. If successful, the modified method computes a partial variational equilibrium characterized by the property that some shared constraints are imposed to have common multipliers across the players while other are not so imposed. Trajectories arising from regularizing the LCP formulations of AGNEPs are shown to converge to a particular type of GNE more general than Rosen’s normalized equilibrium that in turn includes a variational equilibrium as a special case. A third avenue for constructing alternate GNE arises from employing a novel constraint reformulation and parameterization technique. The associated parametric solution method is capable of identifying continuous manifolds of equilibria. Numerical results suggest that the modified Lemke’s method is more robust than the standard version of the method and entails only a modest increase in computational effort on the problems tested. Finally, we show that the conditions for applying the modified Lemke’s scheme are readily satisfied in a breadth of application problems drawn from communication networks, environmental pollution games, and power markets.

The quasi-Newton strategy presented in this paper preserves one of the most important features of the stabilized Sequential Quadratic Programming (sSQP) method, the local convergence without constraint qualifications assumptions. It is known that the primal-dual sequence converges quadratically assuming only the second-order sufficient condition. In this work, we show that if the matrices are updated by performing a minimization of a Bregman distance (which includes the classic updates), the quasi-Newton version of the method converges superlinearly without introducing further assumptions. Also, we show that even for an unbounded Lagrange multiplier set, the generated matrices satisfies a bounded deterioration property and the Dennis-Moré condition. Fil: Fernández Ferreyra, Damián Roberto. Universidad Nacional de Cordoba. Facultad de Matematica, Astronomia y Fisica. Seccion Matematica; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Córdoba; Argentina

Due to their fundamental nature and numerous applications, sphere constrained polynomial optimization problems have received a lot of attention lately. In this paper, we consider three such problems: (i) maximizing a homogeneous polynomial over the sphere; (ii) maximizing a multilinear form over a Cartesian product of spheres; and (iii) maximizing a multiquadratic form over a Cartesian product of spheres. Since these problems are generally intractable, our focus is on designing polynomial-time approximation algorithms for them. By reducing the above problems to that of determining the L 2-diameters of certain convex bodies, we show that they can all be approximated to within a factor of Ω((log n/n) d/2–1) deterministically, where n is the number of variables and d is the degree of the polynomial. This improves upon the currently best known approximation bound of Ω((1/n) d/2–1) in the literature. We believe that our approach will find further applications in the design of approximation algorithms for polynomial optimization problems with provable guarantees.

At the intersection of nonlinear and combinatorial optimization, quadratic programming has attracted significant interest over the past several decades. A variety of relaxations for quadratically constrained quadratic programming (QCQP) can be formulated as semidefinite programs (SDPs). The primary purpose of this paper is to present a systematic comparison of SDP relaxations for QCQP. Using theoretical analysis, it is shown that the recently developed doubly nonnegative relaxation is equivalent to the Shor relaxation, when the latter is enhanced with a partial first-order relaxation-linearization technique. These two relaxations are shown to theoretically dominate six other SDP relaxations. A computational comparison reveals that the two dominant relaxations require three orders of magnitude more computational time than the weaker relaxations, while providing relaxation gaps averaging 3% as opposed to gaps of up to 19% for weaker relaxations, on 700 randomly generated problems with up to 60 variables. An SDP relaxation derived from Lagrangian relaxation, after the addition of redundant nonlinear constraints to the primal, achieves gaps averaging 13% in a few CPU seconds.