Search

This paper introduces the design and implementation of two parallel dual simplex solvers for general large scale sparse linear programming problems. One approach, called PAMI, extends a relatively unknown pivoting strategy called suboptimization and exploits parallelism across multiple iterations. The other, called SIP, exploits purely single iteration parallelism by overlapping computational components when possible. Computational results show that the performance of PAMI is superior to that of the leading open-source simplex solver, and that SIP complements PAMI in achieving speedup when PAMI results in slowdown. One of the authors has implemented the techniques underlying PAMI within the FICO Xpress simplex solver and this paper presents computational results demonstrating their value. This performance increase is sufficiently valuable for the achievement to be used as the basis of promotional material by FICO. In developing the first parallel revised simplex solver of general utility and commercial importance, this work represents a significant achievement in computational optimization., Comment: 27 pages

Recent years have witnessed the surge of asynchronous parallel (async-parallel) iterative algorithms due to problems involving very large-scale data and a large number of decision variables. Because of asynchrony, the iterates are computed with outdated information, and the age of the outdated information, which we call delay, is the number of times it has been updated since its creation. Almost all recent works prove convergence under the assumption of a finite maximum delay and set their stepsize parameters accordingly. However, the maximum delay is practically unknown. This paper presents convergence analysis of an async-parallel method from a probabilistic viewpoint, and it allows for large unbounded delays. An explicit formula of stepsize that guarantees convergence is given depending on delays' statistics. With $p+1$ identical processors, we empirically measured that delays closely follow the Poisson distribution with parameter $p$, matching our theoretical model, and thus the stepsize can be set accordingly. Simulations on both convex and nonconvex optimization problems demonstrate the validness of our analysis and also show that the existing maximum-delay induced stepsize is too conservative, often slowing down the convergence of the algorithm., Comment: accepted to JORSC

We prove new necessary and sufficient conditions to carry out a compact linearization approach for a general class of binary quadratic problems subject to assignment constraints as it has been proposed by Liberti in 2007. The new conditions resolve inconsistencies that can occur when the original method is used. We also present a mixed-integer linear program to compute a minimally-sized linearization. When all the assignment constraints have non-overlapping variable support, this program is shown to have a totally unimodular constraint matrix. Finally, we give a polynomial-time combinatorial algorithm that is exact in this case and can still be used as a heuristic otherwise., Comment: typos fixed, reference added

Block Coordinate Update (BCU) methods enjoy low per-update computational complexity because every time only one or a few block variables would need to be updated among possibly a large number of blocks. They are also easily parallelized and thus have been particularly popular for solving problems involving large-scale dataset and/or variables. In this paper, we propose a primal-dual BCU method for solving linearly constrained convex program in multi-block variables. The method is an accelerated version of a primal-dual algorithm proposed by the authors, which applies randomization in selecting block variables to update and establishes an $O(1/t)$ convergence rate under weak convexity assumption. We show that the rate can be accelerated to $O(1/t^2)$ if the objective is strongly convex. In addition, if one block variable is independent of the others in the objective, we then show that the algorithm can be modified to achieve a linear rate of convergence. The numerical experiments show that the accelerated method performs stably with a single set of parameters while the original method needs to tune the parameters for different datasets in order to achieve a comparable level of performance., Accepted to Computational Optimization and Applications

The alternating direction method of multipliers (ADMM) has been successfully applied to solve structured convex optimization problems due to its superior practical performance. The convergence properties of the 2-block ADMM have been studied extensively in the literature. Specifically, it has been proven that the 2-block ADMM globally converges for any penalty parameter $$\gamma >0$$ . In this sense, the 2-block ADMM allows the parameter to be free, i.e., there is no need to restrict the value for the parameter when implementing this algorithm in order to ensure convergence. However, for the 3-block ADMM, Chen et al. (Math Program 155:57–79, 2016) recently constructed a counter-example showing that it can diverge if no further condition is imposed. The existing results on studying further sufficient conditions on guaranteeing the convergence of the 3-block ADMM usually require $$\gamma $$ to be smaller than a certain bound, which is usually either difficult to compute or too small to make it a practical algorithm. In this paper, we show that the 3-block ADMM still globally converges with any penalty parameter $$\gamma >0$$ if the third function $$f_3$$ in the objective is smooth and strongly convex, and its condition number is in [1, 1.0798), besides some other mild conditions. This requirement covers an important class of problems to be called regularized least squares decomposition (RLSD) in this paper.

A general, inexact, efficient proximal quasi-Newton algorithm for composite optimization problems has been proposed by Scheinberg and Tang (Math Program 160:495–529, 2016) and a sublinear global convergence rate has been established. In this paper, we analyze the global convergence rate of this method, in the both exact and inexact settings, in the case when the objective function is strongly convex. We also investigate a practical variant of this method by establishing a simple stopping criterion for the subproblem optimization. Furthermore, we consider an accelerated variant, based on FISTA of Beck and Teboulle (SIAM 2:183–202, 2009), to the proximal quasi-Newton algorithm. Jiang et al. (SIAM 22:1042–1064, 2012) considered a similar accelerated method, where the convergence rate analysis relies on very strong impractical assumptions on Hessian estimates. We present a modified analysis while relaxing these assumptions and perform a numerical comparison of the accelerated proximal quasi-Newton algorithm and the regular one. Our analysis and computational results show that acceleration may not bring any benefit in the quasi-Newton setting.

A framework is developed for applying accelerated methods to general hyperbolic programming, including linear, second-order cone, and semidefinite programming as special cases. The approach replaces a hyperbolic program with a convex optimization problem whose smooth objective function is explicit, and for which the only constraints are linear equations (one more linear equation than for the original problem). Virtually any first-order method can be applied. Iteration bounds for a representative accelerated method are derived., Comment: A (serious) typo in specifying the main algorithm has been corrected, and suggestions made by referees have been addressed (submitted to Mathematical Programming)

We treat uncertain linear programming problems by utilizing the notion of weighted analytic centers and notions from the area of multi-criteria decision making. After introducing our approach, we develop interactive cutting-plane algorithms for robust optimization, based on concave and quasi-concave utility functions. In addition to practical advantages, due to the flexibility of our approach, we prove that under a theoretical framework proposed by Bertsimas and Sim (Oper Res 52:35–53, 2004), which establishes the existence of certain convex formulation of robust optimization problems, the robust optimal solutions generated by our algorithms are at least as desirable to the decision maker as any solution generated by many other robust optimization algorithms in the theoretical framework. We present some probabilistic bounds for feasibility of robust solutions and evaluate our approach by means of computational experiments.

We study a Lagrangian decomposition algorithm recently proposed by Dan Bienstock and Mark Zuckerberg for solving the LP relaxation of a class of open pit mine project scheduling problems. In this study we show that the Bienstock---Zuckerberg (BZ) algorithm can be used to solve LP relaxations corresponding to a much broader class of scheduling problems, including the well-known Resource Constrained Project Scheduling Problem (RCPSP), and multi-modal variants of the RCPSP that consider batch processing of jobs. We present a new, intuitive proof of correctness for the BZ algorithm that works by casting the BZ algorithm as a column generation algorithm. This analysis allows us to draw parallels with the well-known Dantzig---Wolfe decomposition (DW) algorithm. We discuss practical computational techniques for speeding up the performance of the BZ and DW algorithms on project scheduling problems. Finally, we present computational experiments independently testing the effectiveness of the BZ and DW algorithms on different sets of publicly available test instances. Our computational experiments confirm that the BZ algorithm significantly outperforms the DW algorithm for the problems considered. Our computational experiments also show that the proposed speed-up techniques can have a significant impact on the solve time. We provide some insights on what might be explaining this significant difference in performance.

We study the polyhedral structure of the static probabilistic lot-sizing problem and propose valid inequalities that integrate information from the chance constraint and the binary setup variables. We prove that the proposed inequalities subsume existing inequalities for this problem, and they are facet-defining under certain conditions. In addition, we show that they give the convex hull description of a related stochastic lot-sizing problem. We propose a new formulation that exploits the simple recourse structure, which significantly reduces the number of variables and constraints of the deterministic equivalent program. This reformulation can be applied to general chance-constrained programs with simple recourse. The computational results show that the proposed inequalities and the new formulation are effective for the the static probabilistic lot-sizing problems.

In this paper, we study the strength of Chvatal---Gomory (CG) cuts and more generally aggregation cuts for packing and covering integer programs (IPs). Aggregation cuts are obtained as follows: given an IP formulation, we first generate a single implied inequality using aggregation of the original constraints, then obtain the integer hull of the set defined by this single inequality with variable bounds, and finally use the inequalities describing the integer hull as cutting-planes. Our first main result is to show that for packing and covering IPs, the CG and aggregation closures can be 2-approximated by simply generating the respective closures for each of the original formulation constraints, without using any aggregations. On the other hand, we use computational experiments to show that aggregation cuts can be arbitrarily stronger than cuts from individual constraints for general IPs. The proof of the above stated results for the case of covering IPs with bounds require the development of some new structural results, which may be of independent interest. Finally, we examine the strength of cuts based on k different aggregation inequalities simultaneously, the so-called multi-row cuts, and show that every packing or covering IP with a large integrality gap also has a large k-aggregation closure rank. In particular, this rank is always at least of the order of the logarithm of the integrality gap.

The aim of this paper is twofold: first, to extend the area of applications of tropical optimization by solving new constrained location problems, and second, to offer new closed-form solutions to general problems that are of interest to location analysis. We consider a constrained minimax single-facility location problem with addends on the plane with rectilinear distance. The solution commences with the representation of the problem in a standard form, and then in terms of tropical mathematics, as a constrained optimization problem. We use a transformation technique, which can act as a template to handle optimization problems in other application areas, and hence is of independent interest. To solve the constrained optimization problem, we apply methods and results of tropical optimization, which provide direct, explicit solutions. The results obtained serve to derive new solutions of the location problem, and of its special cases with reduced sets of constraints, in a closed form, ready for practical implementation and immediate computation. As illustrations, numerical solutions of example problems and their graphical representation are given. We conclude with an application of the results to optimal location of the central monitoring facility in an indoor video surveillance system in a multi-floor building environment., 29 pages, 3 figures

In this paper we present a new iterative projection method for finding the closest point in the intersection of convex sets to any arbitrary point in a Hilbert space. This method, termed AAMR for averaged alternating modified reflections, can be viewed as an adequate modification of the Douglas–Rachford method that yields a solution to the best approximation problem. Under a constraint qualification at the point of interest, we show strong convergence of the method. In fact, the so-called strong CHIP fully characterizes the convergence of the AAMR method for every point in the space. We report some promising numerical experiments where we compare the performance of AAMR against other projection methods for finding the closest point in the intersection of pairs of finite dimensional subspaces. This work was partially supported by MINECO of Spain and ERDF of EU, Grant MTM2014-59179-C2-1-P. F.J. Aragón Artacho was supported by the Ramón y Cajal program by MINECO of Spain and ERDF of EU (RYC-2013-13327) and R. Campoy was supported by MINECO of Spain and ESF of EU (BES-2015-073360) under the program “Ayudas para contratos predoctorales para la formación de doctores 2015”.

A bilevel hierarchical clustering model is commonly used in designing optimal multicast networks. In this paper, we consider two different formulations of the bilevel hierarchical clustering problem, a discrete optimization problem which can be shown to be NP-hard. Our approach is to reformulate the problem as a continuous optimization problem by making some relaxations on the discreteness conditions. Then Nesterov's smoothing technique and a numerical algorithm for minimizing differences of convex functions called the DCA are applied to cope with the nonsmoothness and nonconvexity of the problem. Numerical examples are provided to illustrate our method.

Variational inequality problems allow for capturing an expansive class of problems, including convex optimization problems, convex Nash games and economic equilibrium problems, amongst others. Yet in most practical settings, such problems are complicated by uncertainty, motivating the examination of a stochastic generalization of the variational inequality problem and its extensions in which the components of the mapping contain expectations. When the associated sets are unbounded, ascertaining existence requires having access to analytical forms of the expectations. Naturally, in practical settings, such expressions are often difficult to derive, severely limiting the applicability of such an approach. Consequently, our goal lies in developing techniques that obviate the need for integration and our emphasis lies in developing tractable and verifiable sufficiency conditions for claiming existence. We begin by recapping almost-sure sufficiency conditions for stochastic variational inequality problems with single-valued maps provided in our prior work Ravat and Shanbhag (in: Proceedings of the American Control Conference (ACC), 2010), Ravat and Shanbhag (SIAM J Optim 21: 1168–1199, 2011) and provide extensions to multi-valued mappings. Next, we extend these statements to quasi-variational regimes where maps can be either single or set-valued. Finally, we refine the obtained results to accommodate stochastic complementarity problems where the maps are either general or co-coercive. The applicability of our results is demonstrated on practically occuring instances of stochastic quasi-variational inequality problems and stochastic complementarity problems, arising as nonsmooth generalized Nash-Cournot games and power markets, respectively.

We introduce and study a geometric modification of the Douglas-Rach\-ford method called the Circumcentered-Douglas-Rachford method. This method iterates by taking the intersection of bisectors of reflection steps for solving certain classes of feasibility problems. The convergence analysis is established for best approximation problems involving two (affine) subspaces and both our theoretical and numerical results compare favorably to the original Douglas-Rachford method. Under suitable conditions, it is shown that the linear rate of convergence of the Circumcentered-Douglas-Rachford method is at least the cosine of the Friedrichs angle between the (affine) subspaces, which is known to be the sharp rate for the Douglas-Rachford method. We also present a preliminary discussion on the Circumcentered-Douglas-Rachford method applied to the many set case and to examples featuring non-affine convex sets.

Many problems in nonlinear analysis and optimization, among them variational inequalities and minimization of convex functions, can be reduced to finding zeros (namely, roots) of set-valued operators. Hence numerous algorithms have been devised in order to achieve this task. A lot of these algorithms are inexact in the sense that they allow perturbations to appear during the iterative process, and hence they enable one to better deal with noise and computational errors, as well as superiorization. For many years a certain fundamental question has remained open regarding many of these known inexact algorithmic schemes in various finite and infinite dimensional settings, namely whether there exist sequences satisfying these inexact schemes when errors appear. We provide a positive answer to this question. Our results also show that various theorems discussing the convergence of these inexact schemes have a genuine merit beyond the exact case. As a by-product we solve the standard and the strongly implicit inexact resolvent inclusion problems, introduce a promising class of functions (fully Legendre functions), establish continuous dependence (stability) properties of the solution of the inexact resolvent inclusion problem and continuity properties of the protoresolvent, and generalize the notion of strong monotonicity., Comment: To appear in Rendiconti del Circolo Matematico di Palermo Series 2; several remarks which discuss fully Legendre functions were moved from Section 13 to Section 3 and were improved slightly; correction of minor inaccuracies, mainly ones related to some references; added DOI and one additional reference

In this paper, we study the Kurdyka-{\L}ojasiewicz (KL) exponent, an important quantity for analyzing the convergence rate of first-order methods. Specifically, we develop various calculus rules to deduce the KL exponent of new (possibly nonconvex and nonsmooth) functions formed from functions with known KL exponents. In addition, we show that the well-studied Luo-Tseng error bound together with a mild assumption on the separation of stationary values implies that the KL exponent is $\frac12$. The Luo-Tseng error bound is known to hold for a large class of concrete structured optimization problems, and thus we deduce the KL exponent of a large class of functions whose exponents were previously unknown. Building upon this and the calculus rules, we are then able to show that for many convex or nonconvex optimization models for applications such as sparse recovery, their objective function's KL exponent is $\frac12$. This includes the least squares problem with smoothly clipped absolute deviation (SCAD) regularization or minimax concave penalty (MCP) regularization and the logistic regression problem with $\ell_1$ regularization. Since many existing local convergence rate analysis for first-order methods in the nonconvex scenario relies on the KL exponent, our results enable us to obtain explicit convergence rate for various first-order methods when they are applied to a large variety of practical optimization models. Finally, we further illustrate how our results can be applied to establishing local linear convergence of the proximal gradient algorithm and the inertial proximal algorithm with constant step-sizes for some specific models that arise in sparse recovery., Comment: The paper has been published in Foundations of Computational Mathematics: https://link.springer.com/article/10.1007/s10208-017-9366-8. In this update, we added the domain and range of g and h in Theorem 3.5 to remove ambiguity. See also the notes under the previous two updates for changes after the journal version is published

We consider a variety of NP-Complete network connectivity problems. We introduce a novel dual-based approach to approximating network design problems with cut-based linear programming relaxations. This approach gives a $3/2$-approximation to Minimum 2-Edge-Connected Spanning Subgraph that is equivalent to a previously proposed algorithm. One well-studied branch of network design models ad hoc networks where each node can either operate at high or low power. If we allow unidirectional links, we can formalize this into the problem Dual Power Assignment (DPA). Our dual-based approach gives a $3/2$-approximation to DPA, improving the previous best approximation known of $11/7\approx 1.57$. Another standard network design problem is Minimum Strongly Connected Spanning Subgraph (MSCS). We propose a new problem generalizing MSCS and DPA called Star Strong Connectivity (SSC). Then we show that our dual-based approach achieves a 1.6-approximation ratio on SSC. As a consequence of our dual-based approximations, we prove new upper bounds on the integrality gaps of these problems., 7/20/2017: Changed Title to be more accurate. Improved presentation and clarity throughout the document (i.e. adding references and fixing typos)

In the context of linear inverse problems, we propose and study a general iterative regularization method allowing to consider large classes of regularizers and data-fit terms. The algorithm we propose is based on a primal-dual diagonal {descent} method. Our analysis establishes convergence as well as stability results. Theoretical findings are complemented with numerical experiments showing state of the art performances., 41 pages, 13 figures. 4-pages version of the paper available at http://opt-ml.org/papers/OPT2016_paper_19.pdf

We introduce two new algorithms to minimise smooth difference of convex (DC) functions that accelerate the convergence of the classical DC algorithm (DCA). We prove that the point computed by DCA can be used to define a descent direction for the objective function evaluated at this point. Our algorithms are based on a combination of DCA together with a line search step that uses this descent direction. Convergence of the algorithms is proved and the rate of convergence is analysed under the Łojasiewicz property of the objective function. We apply our algorithms to a class of smooth DC programs arising in the study of biochemical reaction networks, where the objective function is real analytic and thus satisfies the Łojasiewicz property. Numerical tests on various biochemical models clearly show that our algorithms outperform DCA, being on average more than four times faster in both computational time and the number of iterations. Numerical experiments show that the algorithms are globally convergent to a non-equilibrium steady state of various biochemical networks, with only chemically consistent restrictions on the network topology. F. J. Aragón Artacho was supported by MINECO of Spain and ERDF of EU, as part of the Ramón y Cajal program (RYC-2013-13327) and the Grant MTM2014-59179-C2-1-P. R. M. Fleming and P. T. Vuong were supported by the U.S. Department of Energy, Offices of Advanced Scientific Computing Research and the Biological and Environmental Research as part of the Scientific Discovery Through Advanced Computing program, Grant #DE-SC0010429.

We provide dual sufficient conditions for subtransversality of collections of sets in an Asplund space setting. For the convex case, we formulate a necessary and sufficient dual criterion of subtransversality in general Banach spaces. Our more general results suggest an intermediate notion of subtransversality, what we call weak intrinsic subtransversality, which lies between intrinsic transversality and subtransversality in Asplund spaces., Comment: 23 pages

We consider stochastic programs where the distribution of the uncertain parameters is only observable through a finite training dataset. Using the Wasserstein metric, we construct a ball in the space of (multivariate and non-discrete) probability distributions centered at the uniform distribution on the training samples, and we seek decisions that perform best in view of the worst-case distribution within this Wasserstein ball. The state-of-the-art methods for solving the resulting distributionally robust optimization problems rely on global optimization techniques, which quickly become computationally excruciating. In this paper we demonstrate that, under mild assumptions, the distributionally robust optimization problems over Wasserstein balls can in fact be reformulated as finite convex programs---in many interesting cases even as tractable linear programs. Leveraging recent measure concentration results, we also show that their solutions enjoy powerful finite-sample performance guarantees. Our theoretical results are exemplified in mean-risk portfolio optimization as well as uncertainty quantification., 42 pages, 10 figures

DIviding RECTangles (DIRECT) is an efficient and popular method in dealing with bound constrained optimization problems. However, DIRECT suffers from dimension curse, since its computational complexity soars when dimension increases. Besides, DIRECT also converges slowly when the objective function is flat. In this paper, we propose a coordinate DIRECT algorithm, which coincides with the spirits of other coordinate update algorithms. We transform the original problem into a series of sub-problems, where only one or several coordinates are selected to optimize and the rest keeps fixed. For each sub-problem, coordinately dividing the feasible domain enjoys low computational burden. Besides, we develop adaptive schemes to keep the efficiency and flexibility to tackle different functions. Specifically, we use block coordinate update, of which the size could be adaptively selected, and we also employ sequential quadratic programming (SQP) to conduct the local search to efficiently accelerate the convergence even when the objective function is flat. With these techniques, the proposed algorithm achieves promising performance on both efficiency and accuracy in numerical experiments.

Motivated by recent work of Renegar, we present new computational methods and associated computational guarantees for solving convex optimization problems using first-order methods. Our problem of interest is the general convex optimization problem $f^* = \min_{x \in Q} f(x)$, where we presume knowledge of a strict lower bound $f_{\mathrm{slb}} < f^*$. [Indeed, $f_{\mathrm{slb}}$ is naturally known when optimizing many loss functions in statistics and machine learning (least-squares, logistic loss, exponential loss, total variation loss, etc.) as well as in Renegar's transformed version of the standard conic optimization problem; in all these cases one has $f_{\mathrm{slb}} = 0 < f^*$.] We introduce a new functional measure called the growth constant $G$ for $f(\cdot)$, that measures how quickly the level sets of $f(\cdot)$ grow relative to the function value, and that plays a fundamental role in the complexity analysis. When $f(\cdot)$ is non-smooth, we present new computational guarantees for the Subgradient Descent Method and for smoothing methods, that can improve existing computational guarantees in several ways, most notably when the initial iterate $x^0$ is far from the optimal solution set. When $f(\cdot)$ is smooth, we present a scheme for periodically restarting the Accelerated Gradient Method that can also improve existing computational guarantees when $x^0$ is far from the optimal solution set, and in the presence of added structure we present a scheme using parametrically increased smoothing that further improves the associated computational guarantees., 1 figure

This paper studies tensor eigenvalue complementarity problems. Basic properties of standard and complementarity tensor eigenvalues are discussed. We formulate tensor eigenvalue complementarity problems as constrained polynomial optimization. When one tensor is strictly copositive, the complementarity eigenvalues can be computed by solving polynomial optimization with normalization by strict copositivity. When no tensor is strictly copositive, we formulate the tensor eigenvalue complementarity problem equivalently as polynomial optimization by a randomization process. The complementarity eigenvalues can be computed sequentially. The formulated polynomial optimization can be solved by Lasserre's hierarchy of semidefinite relaxations. We show that it has finite convergence for general tensors. Numerical experiments are presented to show the efficiency of proposed methods., Comment: 26 pages

A $k$-submodular function is a generalization of submodular and bisubmodular functions. This paper establishes a compact representation for minimizers of a $k$-submodular function by a poset with inconsistent pairs (PIP). This is a generalization of Ando-Fujishige's signed poset representation for minimizers of a bisubmodular function. We completely characterize the class of PIPs (elementary PIPs) arising from $k$-submodular functions. We give algorithms to construct the elementary PIP of minimizers of a $k$-submodular function $f$ for three cases: (i) a minimizing oracle of $f$ is available, (ii) $f$ is network-representable, and (iii) $f$ arises from a Potts energy function. Furthermore, we provide an efficient enumeration algorithm for all maximal minimizers of a Potts $k$-submodular function. Our results are applicable to obtain all maximal persistent labelings in actual computer vision problems. We present experimental results for real vision instances., An earlier version of this paper was presented at the 4th International Symposium on Combinatorial Optimization (ISCO 2016), Vietri sul Mare, Italy, May 16--18, 2016

In this paper we carry out an asymptotic analysis of the proximal-gradient dynamical system \begin{equation*}\left\{ \begin{array}{ll} \dot x(t) +x(t) = \prox_{\gamma f}\big[x(t)-\gamma\nabla\Phi(x(t))-ax(t)-by(t)\big],\\ \dot y(t)+ax(t)+by(t)=0 \end{array}\right.\end{equation*} where $f$ is a proper, convex and lower semicontinuous function, $\Phi$ a possibly nonconvex smooth function and $\gamma, a$ and $b$ are positive real numbers. We show that the generated trajectories approach the set of critical points of $f+\Phi$, here understood as zeros of its limiting subdifferential, under the premise that a regularization of this sum function satisfies the Kurdyka-\L{}ojasiewicz property. We also establish convergence rates for the trajectories, formulated in terms of the \L{}ojasiewicz exponent of the considered regularization function., Comment: arXiv admin note: substantial text overlap with arXiv:1507.01416

We propose a two-step algorithm for optimal controlled islanding that partitions a power grid into islands of limited volume while optimizing several criteria: high generator coherency inside islands, minimum power flow disruption due to teared lines, and minimum load shedding. Several spectral clustering strategies are used in the first step to lower the problem dimension (taking into account coherency and disruption only), and CPLEX tools for the mixed-integer quadratic problem are employed in the second step to choose a balanced partition of the aggregated grid that minimizes a combination of coherency, disruption and load shedding. A greedy heuristic efficiently limits search space by generating starting solution for exact algorithm. Dimension of the second-step problem depends only on the desired number of islands K instead of the dimension of the original grid. The algorithm is tested on standard systems with 118, 2383, and 9241 nodes showing high quality of partitions and competitive computation time., 37 pages, 13 figures, accepted to Energy Systems

In the context of global optimization and mixed-integer non-linear programming, generalizing a technique of D’Ambrosio, Fampa, Lee and Vigerske for handling the square-root function, we develop a virtuous smoothing method, using cubics, aimed at functions having some limited non-smoothness. Our results pertain to root functions ( $$w^p$$ with $$0

We investigate in this paper the generalized trust region subproblem (GTRS) of minimizing a general quadratic objective function subject to a general quadratic inequality constraint. By applying a simultaneous block diagonalization approach, we obtain a congruent canonical form for the symmetric matrices in both the objective and constraint functions. By exploiting the block separability of the canonical form, we show that all GTRSs with an optimal value bounded from below are second order cone programming (SOCP) representable. Our result generalizes the recent work of Ben-Tal and Hertog (Math. Program. 143(1-2):1-29, 2014), which establishes the SOCP representability of the GTRS under the assumption of the simultaneous diagonalizability of the two matrices in the objective and constraint functions. Compared with the state-of-the-art approach to reformulate the GTRS as a semi-definite programming problem, our SOCP reformulation delivers a much faster solution algorithm. We further extend our method to two variants of the GTRS in which the inequality constraint is replaced by either an equality constraint or an interval constraint. Our methods also enable us to obtain simplified versions of the classical S-lemma, the S-lemma with equality, and the S-lemma with interval bounds.

A class of non-convex optimization problems with DC objective function is studied, where DC stands for being representable as the difference $f=g-h$ of two convex functions $g$ and $h$. In particular, we deal with the special case where one of the two convex functions $g$ or $h$ is polyhedral. In case $g$ is polyhedral, we show that a solution of the DC program can be obtained from a solution of an associated polyhedral projection problem. In case $h$ is polyhedral, we prove that a solution of the DC program can be obtained by solving a polyhedral projection problem and finitely many convex programs. Since polyhedral projection is equivalent to multiple objective linear programming (MOLP), a MOLP solver (in the second case together with a convex programming solver) can be used to solve instances of DC programs with polyhedral component. Numerical examples are provided, among them an application to locational analysis., Comment: 2nd version: some typos corrected, some remarks added, update of references

This paper shows that the OSGA algorithm -- which uses first-order information to solve convex optimization problems with optimal complexity -- can be used to efficiently solve arbitrary bound-constrained convex optimization problems. This is done by constructing an explicit method as well as an inexact scheme for solving the bound-constrained rational subproblem required by OSGA. This leads to an efficient implementation of OSGA on large-scale problems in applications arising signal and image processing, machine learning and statistics. Numerical experiments demonstrate the promising performance of OSGA on such problems. ions to show the efficiency of the proposed scheme. A software package implementing OSGA for bound-constrained convex problems is available.

We study connections between optimistic bilevel programming problems and generalized Nash equilibrium problems. We remark that, with respect to bilevel problems, we consider the general case in which the lower level program is not assumed to have a unique solution. Inspired by the optimal value approach, we propose a Nash game that, transforming the so-called implicit value function constraint into an explicitly defined constraint function, incorporates some taste of hierarchy and turns out to be related to the bilevel programming problem. We provide a complete theoretical analysis of the relationship between the vertical bilevel problem and our ``uneven'' horizontal model: in particular, we define classes of problems for which solutions of the bilevel program can be computed by finding equilibria of our game. Furthermore, by referring to some applications in economics, we show that our ``uneven'' horizontal model, in some sense, lies between the vertical bilevel model and a ``pure'' horizontal game.

In conic linear programming -- in contrast to linear programming -- the Lagrange dual is not an exact dual: it may not attain its optimal value, or there may be a positive duality gap. The corresponding Farkas' lemma is also not exact (it does not always prove infeasibility). We describe exact duals, and certificates of infeasibility and weak infeasibility for conic LPs which are nearly as simple as the Lagrange dual, but do not rely on any constraint qualification. Some of our exact duals generalize the SDP duals of Ramana, and Klep and Schweighofer to the context of general conic LPs. Some of our infeasibility certificates generalize the row echelon form of a linear system of equations: they consist of a small, trivially infeasible subsystem obtained by elementary row operations. We prove analogous results for weakly infeasible systems. We obtain some fundamental geometric corollaries: an exact characterization of when the linear image of a closed convex cone is closed, and an exact characterization of nice cones. Our infeasibility certificates provide algorithms to generate {\em all} infeasible conic LPs over several important classes of cones; and {\em all} weakly infeasible SDPs in a natural class. Using these algorithms we generate a public domain library of infeasible and weakly infeasible SDPs. The status of our instances can be verified by inspection in exact arithmetic, but they turn out to be challenging for commercial and research codes., Comment: Fixed the last few typos. To appear in Mathematical Programming, Series A

We consider the NP-hard problem of minimizing a convex quadratic function over the integer lattice ${\bf Z}^n$. We present a simple semidefinite programming (SDP) relaxation for obtaining a nontrivial lower bound on the optimal value of the problem. By interpreting the solution to the SDP relaxation probabilistically, we obtain a randomized algorithm for finding good suboptimal solutions, and thus an upper bound on the optimal value. The effectiveness of the method is shown for numerical problem instances of various sizes., 25 pages, 3 figures; to appear in Optimization Letters (OPTL)

The main goal of this paper is to present results of existence and non-existence of convex functions on Riemannian manifolds and, in the case of the existence, we associate such functions to the geometry of the manifold. Precisely, we prove that the conservativity of the geodesic flow on a Rieman- nain manifold with infinite volume is an obstruction to the existence of convex functions. Next, we present a geometric condition that ensures the existence of (strictly) convex functions on a particular class of complete non-compact man- ifolds, and, we use this fact to construct a manifold whose sectional curvature assumes any real value greater than a negative constant and admits a strictly convex function. In the last result we relate the geometry of a Riemannian manifold of positive sectional curvature with the set of minimum points of a convex function defined on the manifold.

A classical result in risk measure theory states that every coherent risk measure has a dual representation as the supremum of certain expected value over a risk envelope. We study this topic in more detail. The related issues include: 1. Set operations of risk envelopes and how they change the risk measures, 2. The structure of risk envelopes of popular risk measures, 3. Aversity of risk measures and its impact to risk envelopes, and 4. A connection between risk measures in stochastic optimization and uncertainty sets in robust optimization., Annals of Operations Research, 2017

Computing explicitly the $$\varepsilon $$ź-subdifferential of a proper function amounts to computing the level set of a convex function namely the conjugate minus a linear function. The resulting theoretical algorithm is applied to the the class of (convex univariate) piecewise linear---quadratic functions for which existing numerical libraries allow practical computations. We visualize the results in a primal, dual, and subdifferential views through several numerical examples. We also provide a visualization of the BrOndsted---Rockafellar theorem.

Locating proximal points is a component of numerous minimization algorithms. This work focuses on developing a method to find the proximal point of a convex function at a point, given an inexact oracle. Our method assumes that exact function values are at hand, but exact subgradients are either not available or not useful. We use approximate subgradients to build a model of the objective function, and prove that the method converges to the true prox-point within acceptable tolerance. The subgradient g k used at each step k is such that the distance from g k to the true subdifferential of the objective function at the current iteration point is bounded by some fixed e > 0. The algorithm includes a novel tilt-correct step applied to the approximate subgradient.

This paper introduces a parallel and distributed extension to the alternating direction method of multipliers (ADMM) for solving convex problem: minimize $\sum_{i=1}^N f_i(x_i)$ subject to $\sum_{i=1}^N A_i x_i=c, x_i\in \mathcal{X}_i$. The algorithm decomposes the original problem into N smaller subproblems and solves them in parallel at each iteration. This Jacobian-type algorithm is well suited for distributed computing and is particularly attractive for solving certain large-scale problems. This paper introduces a few novel results. Firstly, it shows that extending ADMM straightforwardly from the classic Gauss-Seidel setting to the Jacobian setting, from 2 blocks to N blocks, will preserve convergence if matrices $A_i$ are mutually near-orthogonal and have full column-rank. Secondly, for general matrices $A_i$, this paper proposes to add proximal terms of different kinds to the N subproblems so that the subproblems can be solved in flexible and efficient ways and the algorithm converges globally at a rate of o(1/k). Thirdly, a simple technique is introduced to improve some existing convergence rates from O(1/k) to o(1/k). In practice, some conditions in our convergence theorems are conservative. Therefore, we introduce a strategy for dynamically tuning the parameters in the algorithm, leading to substantial acceleration of the convergence in practice. Numerical results are presented to demonstrate the efficiency of the proposed method in comparison with several existing parallel algorithms. We implemented our algorithm on Amazon EC2, an on-demand public computing cloud, and report its performance on very large-scale basis pursuit problems with distributed data., Comment: The second version has a new title

This paper considers the problem of unconstrained minimization of smooth convex functions having Lipschitz continuous gradients with known Lipschitz constant. We recently proposed the optimized gradient method for this problem and showed that it has a worst-case convergence bound for the cost function decrease that is twice as small as that of Nesterov's fast gradient method, yet has a similarly efficient practical implementation. Drori showed recently that the optimized gradient method has optimal complexity for the cost function decrease over the general class of first-order methods. This optimality makes it important to study fully the convergence properties of the optimized gradient method. The previous worst-case convergence bound for the optimized gradient method was derived for only the last iterate of a secondary sequence. This paper provides an analytic convergence bound for the primary sequence generated by the optimized gradient method. We then discuss additional convergence properties of the optimized gradient method, including the interesting fact that the optimized gradient method has two types of worst-case functions: a piecewise affine-quadratic function and a quadratic function. These results help complete the theory of an optimal first-order method for smooth convex minimization.

The subdifferential of a function is a generalization for nonsmooth functions of the concept of gradient. It is frequently used in variational analysis, particularly in the context of nonsmooth optimization. The present work proposes algorithms to reconstruct a polyhedral subdifferential of a function from the computation of finitely many directional derivatives. We provide upper bounds on the required number of directional derivatives when the space is $\R^1$ and $\R^2$, as well as in $\R^n$ where subdifferential is known to possess at most three vertices.

Two-trust-region subproblem (TTRS), which is the minimization of a general quadratic function over the intersection of two full-dimensional ellipsoids, has been the subject of several recent research. In this paper, to solve TTRS, a hybrid of efficient algorithms for finding global and local-nonglobal minimizers of trust-region subproblem and the alternating direction method of multipliers (ADMM) is proposed. The convergence of the ADMM steps to the first-order stationary condition is proved under certain conditions. On several test problems, we compare the new algorithm against three competitors: the Snopt software, the algorithm proposed by Sakaue et al. (SIAM J Optim 26:1669–1694, 2016) and the CADMM algorithm proposed by Huang and Sidiropoulos (IEEE Trans Signal Process 64:5297–5310, 2016). The numerical results show that the new algorithm is competitive.

A methodology is introduced which uses three simple objective function features to predict effective control parameters for differential evolution. This is achieved using cluster analysis techniques to classify objective functions using these features. Information on prior performance of various control parameters for each classification is then used to determine which control parameters to use in future optimisations. Our approach is compared to state-of-the-art adaptive and non-adaptive techniques. Two accepted bench mark suites are used to compare performance and in all cases we show that the improvement resulting from our approach is statistically significant. The majority of the computational effort of this methodology is performed off-line, however even when taking into account the additional on-line cost our approach outperforms other adaptive techniques. We also investigate the key tuning parameters of our methodology, such as number of clusters, which further support the finding that the simple features selected are predictors of effective control parameters. The findings presented in this paper are significant because they show that simple to calculate features of objective functions can help to select control parameters for optimisation algorithms. This can have an immediate positive impact on the application of these optimisation algorithms on real world problems, where it is often difficult to select effective control parameters., Cite this article as: Walton, S.P. & Brown, M.R. J Heuristics (2019). https://doi.org/10.1007/s10732-019-09419-8

This paper revisits the well-studied fixed point problem from a unified viewpoint of mathematical modeling and canonical duality theory, i.e., the general fixed point problem is first reformulated as a nonconvex optimization problem, its well-posedness is discussed based on the objectivity principle in continuum physics; then the canonical duality theory is applied for solving this challenging problem to obtain not only all fixed points, but also their stability properties. Applications are illustrated by problems governed by nonconvex polynomial, exponential, and logarithmic operators. This paper shows that within the framework of the canonical duality theory, there is no difference between the fixed point problems and nonconvex analysis/optimization in multidisciplinary studies.

Let $C$ be a proper convex cone generated by a compact set which supports a measure $\mu$. A construction due to A.Barvinok, E.Veomett and J.B. Lasserre produces, using $\mu$, a sequence $(P_k)_{k\in \mathbb{N}}$ of nested spectrahedral cones which contains the cone $C^*$ dual to $C$. We prove convergence results for such sequences of spectrahedra and provide tools for bounding the distance between $P_k$ and $C^*$. These tools are especially useful on cones with enough symmetries and allow us to determine bounds for several cones of interest. We compute such upper bounds for semidefinite approximations of cones over traveling salesman polytopes and for cones of nonnegative ternary sextics and quaternary quartics., Comment: 25 pages, 8 figures

In this article, we study an infinite horizon optimal control problem with monotone controls. We analyze the associated Hamilton–Jacobi–Bellman (HJB) variational inequality which characterizes the value function and consider the totally discretized problem using Lagrange elements to approximate the state space Ω. The convergence orders of these approximations are proved, which are in general (h+kh)γ where γ is the Hölder constant of the value function u, h and k are the time and space discretization parameters, respectively. A special election of the relations between h and k allows to obtain a convergence of order k23γ, which is valid without semiconcavity hypotheses over the problem’s data. Fil: Aragone, Laura Susana. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Rosario. Centro Internacional Franco Argentino de Ciencias de la Información y de Sistemas. Universidad Nacional de Rosario. Centro Internacional Franco Argentino de Ciencias de la Información y de Sistemas; Argentina Fil: Parente, Lisandro Armando. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Rosario. Centro Internacional Franco Argentino de Ciencias de la Información y de Sistemas. Universidad Nacional de Rosario. Centro Internacional Franco Argentino de Ciencias de la Información y de Sistemas; Argentina Fil: Philipp, Eduardo Andrés. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Rosario. Centro Internacional Franco Argentino de Ciencias de la Información y de Sistemas. Universidad Nacional de Rosario. Centro Internacional Franco Argentino de Ciencias de la Información y de Sistemas; Argentina

The decomposition of a matrix, as a product of factors with particular properties, is a much used tool in numerical analysis. Here we develop methods for decomposing a matrix $C$ into a product $X Y$, where the factors $X$ and $Y$ are required to minimize their distance from an arbitrary pair $X_0$ and $Y_0$. This type of decomposition, a projection to a matrix product constraint, in combination with projections that impose structural properties on $X$ and $Y$, forms the basis of a general method of decomposing a matrix into factors with specified properties. Results are presented for the application of these methods to a number of hard problems in exact factorization., Comment: 37 pages, 8 figures

In this paper, we consider a class of $n$-person noncooperative games, where the utility function of every player is given by a homogeneous polynomial defined by the payoff tensor of that player, which is a natural extension of the bimatrix game where the utility function of every player is given by a quadratic form defined by the payoff matrix of that player. We will call such a problem the multilinear game. We reformulate the multilinear game as a tensor complementarity problem, a generalization of the linear complementarity problem; and show that finding a Nash equilibrium point of the multilinear game is equivalent to finding a solution of the resulted tensor complementarity problem. Especially, we present an explicit relationship between the solutions of the multilinear game and the tensor complementarity problem, which builds a bridge between these two classes of problems. We also apply a smoothing-type algorithm to solve the resulted tensor complementarity problem and give some preliminary numerical results for solving the multilinear games.