Search

In this paper, elliptic PDE-constrained optimization problems with box constraints on the control are considered. To numerically solve the problems, we apply the ‘optimize-discretize-optimize’ strategy. Specifically, the alternating direction method of multipliers (ADMM) algorithm is applied in function space first, and then, the standard piecewise linear finite-element approach is employed to discretize the subproblems in each iteration. Finally, some efficient numerical methods are applied to solve the discretized subproblems based on their structures. Motivated by the idea of the multi-level strategy, instead of fixing the mesh size before the computation process, we propose the strategy of gradually refining the grid. Moreover, the subproblems in each iteration are solved by appropriate inexact methods. Based on the strategies above, an efficient convergent multi-level ADMM (mADMM) algorithm is proposed. We present the convergence analysis and the iteration complexity results o(1/k) for the mADMM algorithm. Some numerical experiments are done and the numerical results show the high efficiency of the mADMM algorithm.

We investigate the asymptotic properties of the trajectories generated by a second-order dynamical system with Hessian driven damping and a Tikhonov regularization term in connection with the minimization of a smooth convex function in Hilbert spaces. We obtain fast convergence results for the function values along the trajectories. The Tikhonov regularization term enables the derivation of strong convergence results of the trajectory to the minimizer of the objective function of minimum norm.

The aim of this survey is to present the main important techniques and tools from variational analysis used for first and second order dynamical systems of implicit type for solving monotone inclusions and non-smooth optimization problems. The differential equations are expressed by means of the resolvent (in case of a maximally monotone set valued operator) or the proximal operator for non-smooth functions. The asymptotic analysis of the trajectories generated relies on Lyapunov theory, where the appropriate energy functional plays a decisive role. While the most part of the paper is related to monotone inclusions and convex optimization problems in the variational case, we present also results for dynamical systems for solving non-convex optimization problems, where the Kurdyka-Lojasiewicz property is used., Comment: Survey, 26 pages, to appear in Set Valued and Variational Analysis. arXiv admin note: text overlap with arXiv:1507.01416, arXiv:1411.4442, arXiv:1503.04652

The complexity of Pareto fronts imposes a great challenge on the convergence analysis of multi-objective optimization methods. While most theoretical convergence studies have addressed finite-set and/or discrete problems, others have provided probabilistic guarantees, assumed a total order on the solutions, or studied their asymptotic behaviour. In this paper, we revisit the Tchebycheff weighted method in a hierarchical bandits setting and provide a finite-time bound on the Pareto-compliant additive $\epsilon$-indicator. To the best of our knowledge, this paper is one of few that establish a link between weighted sum methods and quality indicators in finite time., Comment: submitted to Journal of Global Optimization. This article's notation and terminology is based on arXiv:1612.08412

Recent several years have witnessed the surge of asynchronous (async-) parallel computing methods due to the extremely big data involved in many modern applications and also the advancement of multi-core machines and computer clusters. In optimization, most works about async-parallel methods are on unconstrained problems or those with block separable constraints. In this paper, we propose an async-parallel method based on block coordinate update (BCU) for solving convex problems with nonseparable linear constraint. Running on a single node, the method becomes a novel randomized primal-dual BCU with adaptive stepsize for multi-block affinely constrained problems. For these problems, Gauss-Seidel cyclic primal-dual BCU needs strong convexity to have convergence. On the contrary, merely assuming convexity, we show that the objective value sequence generated by the proposed algorithm converges in probability to the optimal value and also the constraint residual to zero. In addition, we establish an ergodic $O(1/k)$ convergence result, where $k$ is the number of iterations. Numerical experiments are performed to demonstrate the efficiency of the proposed method and significantly better speed-up performance than its sync-parallel counterpart.

The strong growth condition (SGC) is known to be a sufficient condition for linear convergence of the stochastic gradient method using a constant step-size $$\gamma $$ (SGM-CS). In this paper, we provide a necessary condition, for the linear convergence of SGM-CS, that is weaker than SGC. Moreover, when this necessary is violated up to a additive perturbation $$\sigma $$ , we show that both the projected stochastic gradient method using a constant step-size, under the restricted strong convexity assumption, and the proximal stochastic gradient method, under the strong convexity assumption, exhibit linear convergence to a noise dominated region, whose distance to the optimal solution is proportional to $${\gamma {\sigma }}$$ .

Proximal gradient methods have been found to be highly effective for solving minimization problems with non-negative constraints or L1-regularization. Under suitable nondegeneracy conditions, it is known that these algorithms identify the optimal sparsity pattern for these types of problems in a finite number of iterations. However, it is not known how many iterations this may take. We introduce the notion of the "active-set complexity", which in these cases is the number of iterations before an algorithm is guaranteed to have identified the final sparsity pattern. We further give a bound on the active-set complexity of proximal gradient methods in the common case of minimizing the sum of a strongly-convex smooth function and a separable convex non-smooth function., Comment: Added discussion of Liang et al. (2017), which previously analyzed active-set complexity of the proximal-gradient (PG) method in the convex setting under an additional assumption. Added convergence analysis of the PG method under a general constant step-size for strongly-convex functions, use this to bound the active-set complexity under a general constant step-size. Additional minor edits

Fuel cells, batteries, thermochemical and other energy conversion devices involve the transport of a number of (electro-)chemical species through distinct materials so that they can meet and react at specified multi-material interfaces. Therefore, morphology or arrangement of these different materials can be critical in the performance of an energy conversion device. In this paper, we study a model problem motivated by a solar-driven thermochemical conversion device that splits water into hydrogen and oxygen. We formulate the problem as a system of coupled multi-material reaction-diffusion equations where each species diffuses selectively through a given material and where the reaction occurs at multi-material interfaces. We express the problem of optimal design of the material arrangement as a saddle point problem and obtain an effective functional which shows that regions with very fine phase mixtures of the material arise naturally. To explore this further, we introduce a phase-field formulation of the optimal design problem, and numerically study selected examples., 25 pages, 10 figures

A (convex) polytope $P$ is said to be $2$-level if for every direction of hyperplanes which is facet-defining for $P$, the vertices of $P$ can be covered with two hyperplanes of that direction. The study of these polytopes is motivated by questions in combinatorial optimization and communication complexity, among others. In this paper, we present the first algorithm for enumerating all combinatorial types of $2$-level polytopes of a given dimension $d$, and provide complete experimental results for $d \leqslant 7$. Our approach is inductive: for each fixed $(d-1)$-dimensional $2$-level polytope $P_0$, we enumerate all $d$-dimensional $2$-level polytopes $P$ that have $P_0$ as a facet. This relies on the enumeration of the closed sets of a closure operator over a finite ground set. By varying the prescribed facet $P_0$, we obtain all $2$-level polytopes in dimension $d$., Comment: 25 pages, 10 figures, 3 tables

An axiomatic approach is applied to the problem of extracting a ranking of the alternatives from a pairwise comparison ratio matrix. The ordering induced by row geometric mean method is proved to be uniquely determined by three independent axioms, anonymity (independence of the labelling of alternatives), responsiveness (a kind of monotonicity property) and aggregation invariance, which requires the preservation of group consensus, that is, the pairwise ranking between two alternatives should remain unchanged if unanimous individual preferences are combined by geometric mean., Comment: 17 pages, 2 figures

We argue that reducing nonlinear programming problems to a simple canonical form is an effective way to analyze them, specially when the problem is degenerate and the usual linear independence hypothesis does not hold. To illustrate this fact we solve an open problem about constraint qualifications using this simple canonical form.

We study the asymptotic value of a frequency-dependent zero-sum game with separable payoff following a differential approach. The stage payoffs in such games depend on the current actions and on a linear function of the frequency of actions played so far. We associate to the repeated game, in a natural way, a differential game and although the latter presents an irregularity at the origin, we prove that it has a value. We conclude, using appropriate approximations, that the asymptotic value of the original game exists in both the n-stage and the {\lambda}-discounted games and that it coincides with the value of the continuous time game., Comment: 22 pages, accepted in Dynamic Games and Applications

This paper investigates new first-order optimality conditions for general optimization problems. These optimality conditions are stronger than the commonly used M-stationarity conditions and are in particular useful when the latter cannot be applied because the underlying limiting normal cone cannot be computed effectively. We apply our optimality conditions to a MPEC to demonstrate their practicability.

We consider global efficiency of algorithms for minimizing a sum of a convex function and a composition of a Lipschitz convex function with a smooth map. The basic algorithm we rely on is the prox-linear method, which in each iteration solves a regularized subproblem formed by linearizing the smooth map. When the subproblems are solved exactly, the method has efficiency $$\mathcal {O}(\varepsilon ^{-2})$$ , akin to gradient descent for smooth minimization. We show that when the subproblems can only be solved by first-order methods, a simple combination of smoothing, the prox-linear method, and a fast-gradient scheme yields an algorithm with complexity $$\widetilde{\mathcal {O}}(\varepsilon ^{-3})$$ . We round off the paper with an inertial prox-linear method that automatically accelerates in presence of convexity.

The main contributions of this paper are the proposition and the convergence analysis of a class of inertial projection-type algorithm for solving variational inequality problems in real Hilbert spaces where the underline operator is monotone and uniformly continuous. We carry out a unified analysis of the proposed method under very mild assumptions. In particular, weak convergence of the generated sequence is established and nonasymptotic $O(1/n)$ rate of convergence is established, where $n$ denotes the iteration counter. We also present some experimental results to illustrate the profits gained by introducing the inertial extrapolation steps., 24 pages, 5 figures. arXiv admin note: substantial text overlap with arXiv:2101.08057

We establish sharp estimates for the convergence rate of the Kranosel'ski\v{\i}-Mann fixed point iteration in general normed spaces, and we use them to show that the asymptotic regularity bound recently proved in [11] (Israel Journal of Mathematics 199(2), 757-772, 2014) with constant $1/\sqrt{\pi}$ is sharp and cannot be improved. To this end we consider the recursive bounds introduced in [3] (Proceedings of the 2nd International Conference on Fixed Point Theory and Applications, World Scientific Press, London, 27-66, 1992) which we reinterpret in terms of a nested family of optimal transport problems. We show that these bounds are tight by building a nonexpansive map $T:[0,1]^{\mathbb N}\to[0,1]^{\mathbb N}$ that attains them with equality, settling the main conjecture in [3]. The recursive bounds are in turn reinterpreted as absorption probabilities for an underlying Markov chain which is used to establish the tightness of the constant $1/\sqrt{\pi}$., Comment: Updated version with minor typos corrections

We consider an optimization problem with positively homogeneous functions in its objective and constraint functions. Examples of such positively homogeneous functions include the absolute value function and the $p$-norm function, where $p$ is a positive real number. The problem, which is not necessarily convex, extends the absolute value optimization proposed in [O. L. Mangasarian, Absolute value programming, Computational Optimization and Applications 36 (2007) pp. 43-53]. In this work, we propose a dual formulation that, differently from the Lagrangian dual approach, has a closed-form and some interesting properties. In particular, we discuss the relation between the Lagrangian duality and the one proposed here, and give some sufficient conditions under which these dual problems coincide. Finally, we show that some well-known problems, e.g., sum of norms optimization and the group Lasso-type optimization problems, can be reformulated as positively homogeneous optimization problems., 21 pages

This paper proposes a strategic model of pollution control. A firm, representative of the productive sector of a country, aims at maximizing its profits by expanding its production. Assuming that the output of production is proportional to the level of pollutants' emissions, the firm increases the level of pollution. The government of the country aims at minimizing the social costs due to the pollution, and introduces regulatory constraints on the emissions' level, which then effectively cap the output of production. Supposing that the firm and the government face both proportional and fixed costs in order to adopt their policies, we model the previous problem as a stochastic impulse two-person nonzero-sum game. The state variable of the game is the level of the output of production which evolves as a general linearly controlled one-dimensional Ito-diffusion. Following an educated guess, we first construct a pair of candidate equilibrium policies and of corresponding equilibrium values, and we then provide a set of sufficient conditions under which they indeed realize an equilibrium. Our results are complemented by a numerical study when the (uncontrolled) output of production evolves as a geometric Brownian motion, and the firm's operating profit and the government's running cost functions are of power type. An analysis of the dependency of the equilibrium policies and values on the model parameters yields interesting new behaviors that we explain as a consequence of the strategic interaction between the firm and the government., Comment: 26 pages; 14 figures; improved presentation of the results and modified a few proofs; Accepted for publication in Annals of Operations Research

We consider a new and general online resource allocation problem, where the goal is to maximize a function of a positive semidefinite (PSD) matrix with a scalar budget constraint. The problem data arrives online, and the algorithm needs to make an irrevocable decision at each step. Of particular interest are classic experiment design problems in the online setting, with the algorithm deciding whether to allocate budget to each experiment as new experiments become available sequentially. We analyze two greedy primal-dual algorithms and provide bounds on their competitive ratios. Our analysis relies on a smooth surrogate of the objective function that needs to satisfy a new diminishing returns (PSD-DR) property (that its gradient is order-reversing with respect to the PSD cone). Using the representation for monotone maps on the PSD cone given by L\"owner's theorem, we obtain a convex parametrization of the family of functions satisfying PSD-DR. We then formulate a convex optimization problem to directly optimize our competitive ratio bound over this set. This design problem can be solved offline before the data start arriving. The online algorithm that uses the designed smoothing is tailored to the given cost function, and enjoys a competitive ratio at least as good as our optimized bound. We provide examples of computing the smooth surrogate for D-optimal and A-optimal experiment design, and demonstrate the performance of the custom-designed algorithm., Comment: 23 pages

This paper continues our recent effort in applying continuous optimization techniques to study optimal multicast communication networks modeled as bilevel hierarchical clustering problems. Given a finite number of nodes, we consider two different models of multicast networks by identifying a certain number of nodes as cluster centers, and at the same time, locating a particular node that serves as a total center so as to minimize the total transportation cost throughout the network. The fact that the cluster centers and the total center have to be among the given nodes makes these problems discrete optimization problems. Our approach is to reformulate the discrete problems as continuous ones and to apply Nesterov's smoothing approximation techniques on the Minkowski gauges that are used as distance measures. This approach enables us to propose two implementable DCA-based algorithms for solving the problems. Numerical results and practical applications are provided to illustrate our approach.

The robust truss topology optimization against the uncertain static external load can be formulated as mixed-integer semidefinite programming. Although a global optimal solution can be computed with a branch-and-bound method, it is very time-consuming. This paper presents an alternative formulation, semidefinite programming with complementarity constraints, and proposes an efficient heuristic. The proposed method is based upon the convex-concave procedure for DC (difference-of-convex) programming. It is shown that the method can often find a practically reasonable truss design within the computational cost of solving some dozen of convex optimization subproblems.

In a previous paper, the authors showed that in a reflexive Banach space the lower limit of a sequence of maximal monotone operators is always representable by a convex function. The present paper gives precisions to the latter result by demonstrating the continuity of the representation with respect to the epi-convergence of the representative functions, and the stability of the class of maximal monotone operators with respect to the Mosco-convergence of their representative functions., 7 pages

The problem of minimizing the rank of a symmetric positive semidefinite matrix subject to constraints can be cast equivalently as a semidefinite program with complementarity constraints (SDCMPCC). The formulation requires two positive semidefinite matrices to be complementary. This is a continuous and nonconvex reformulation of the rank minimization problem. We investigate calmness of locally optimal solutions to the SDCMPCC formulation and hence show that any locally optimal solution is a KKT point. We develop a penalty formulation of the problem. We present calmness results for locally optimal solutions to the penalty formulation. We also develop a proximal alternating linearized minimization (PALM) scheme for the penalty formulation, and investigate the incorporation of a momentum term into the algorithm. Computational results are presented.

With the increasing interest in applying the methodology of difference-of-convex (dc) optimization to diverse problems in engineering and statistics, this paper establishes the dc property of many functions in various areas of applications not previously known to be of this class. Motivated by a quadratic programming based recourse function in two-stage stochastic programming, we show that the (optimal) value function of a copositive (thus not necessarily convex) quadratic program is dc on the domain of finiteness of the program when the matrix in the objective function’s quadratic term and the constraint matrix are fixed. The proof of this result is based on a dc decomposition of a piecewise $$\hbox {LC}^1$$ function (i.e., functions with Lipschitz gradients). Armed with these new results and known properties of dc functions existed in the literature, we show that many composite statistical functions in risk analysis, including the value-at-risk (VaR), conditional value-at-risk (CVaR), optimized certainty equivalent, and the expectation-based, VaR-based, and CVaR-based random deviation functionals are all dc. Adding the known class of dc surrogate sparsity functions that are employed as approximations of the $$\ell _0$$ function in statistical learning, our work significantly expands the classes of dc functions and positions them for fruitful applications.

There are different solution concepts for convex vector optimization problems (CVOPs) and a recent one, which is motivated from a set optimization point of view, consists of finitely many efficient solutions that generate polyhedral inner and outer approximations to the Pareto frontier. A CVOP with compact feasible region is known to be bounded and there exists a solution of this sense to it. However, it is not known if it is possible to generate polyhedral inner and outer approximations to the Pareto frontier of a CVOP if the feasible region is not compact. This study shows that not all CVOPs are tractable in that sense and gives a characterization of tractable problems in terms of the well known weighted sum scalarization problems.

In several multiobjective decision problems Pairwise Comparison Matrices (PCM) are applied to evaluate the decision variants. The problem that arises very often is the inconsistency of a given PCM. In such a situation it is important to approximate the PCM with a consistent one. One of the approaches is to minimize the distance between the matrices, most often the Euclidean distance. In the paper we consider the problem of minimizing the maximum distance. After applying the logarithmic transformation we are able to formulate the obtained subproblem as a Shortest Path Problem and solve it more efficiently. We analyze the structure of the set of optimal solutions and prove some of its properties. This allows us to provide an iterative algorithm that results in a unique, Pareto-efficient solution.

This paper considers a conceptual version of a convex optimization algorithm whic is based on replacing a convex optimization problem with the root-finding problem for the approximate sub-differential mapping which is solved by repeated projection onto the epigraph of conjugate function. Whilst the projection problem is not exactly solvable in finite space-time it can be approximately solved up to arbitrary precision by simple iterative methods, which use linear support functions of the epigraph. It seems therefore useful to study computational characteristics of the idealized version of this algorithm when projection on the epigraph is computed precisely to estimate the potential benefits for such development. The key results of this study are that the conceptual algorithm attains super-linear rate of convergence in general convex case, the rate of convergence becomes quadratic for objective functions forming super-set of strongly convex functions, and convergence is finite when objective function has sharp minimum. In all cases convergence is global and does not require differentiability of the objective. Keywords: convex optimization, conjugate function, approximate sub-differential, super-linear convergence, quadratic convergence, finite convergence, projection, epigraph

In Part 1 of this paper, we have estimated the Fr\'echet coderivative and the Mordukhovich coderivative of the stationary point set map of a smooth parametric optimization problem with one smooth functional constraint under total perturbations. From these estimates, necessary and sufficient conditions for the local Lipschitz-like property of the map have been obtained. In this part, we establish sufficient conditions for the Robinson stability of the stationary point set map. This allows us to revisit and extend several stability theorems in indefinite quadratic programming. A comparison of our results with the ones which can be obtained via another approach is also given., Comment: This manuscript is based on the paper "Sensitivity Analysis of a Stationary Point Set Map under Total Perturbations. Part 2: Robinson Stability" which has been pubplished in Journal of Optimization Theory and Applications (DOI: 10.1007/s10957-018-1295-4). We have added the Section 6 "Appendices" to the paper. This section presents two proofs of Lemmas 5.1 and 5.2

By applying some theorems of Levy and Mordukhovich (Math Program 99: 311--327, 2004) and other related results, we estimate the Fr\'echet coderivative and the Mordukhovich coderivative of the stationary point set map of a smooth parametric optimization problem with one smooth functional constraint under total perturbations. From the obtained formulas we derive necessary and sufficient conditions for the local Lipschitz-like property of the stationary point set map. This leads us to new insights into the preceding deep investigations of Levy and Mordukhovich in the above-cited paper and of Qui (J Optim Theory Appl 161: 398--429, 2014; J Glob Optim 65: 615--635, 2016)., Comment: This paper has been published in Journal of Optimization Theory and Applications

We propose a new family of inexact sequential quadratic approximation (SQA) methods, which we call the inexact regularized proximal Newton (IRPN) method, for minimizing the sum of two closed proper convex functions, one of which is smooth and the other is possibly non-smooth. Our proposed method features strong convergence guarantees even when applied to problems with degenerate solutions while allowing the inner minimization to be solved inexactly. Specifically, we prove that when the problem possesses the so-called Luo–Tseng error bound (EB) property, IRPN converges globally to an optimal solution, and the local convergence rate of the sequence of iterates generated by IRPN is linear, superlinear, or even quadratic, depending on the choice of parameters of the algorithm. Prior to this work, such EB property has been extensively used to establish the linear convergence of various first-order methods. However, to the best of our knowledge, this work is the first to use the Luo–Tseng EB property to establish the superlinear convergence of SQA-type methods for non-smooth convex minimization. As a consequence of our result, IRPN is capable of solving regularized regression or classification problems under the high-dimensional setting with provable convergence guarantees. We compare our proposed IRPN with several empirically efficient algorithms by applying them to the $$\ell _1$$ -regularized logistic regression problem. Experiment results show the competitiveness of our proposed method.

This paper proposes tight semidefinite relaxations for polynomial optimization. The optimality conditions are investigated. We show that generally Lagrange multipliers can be expressed as polynomial functions in decision variables over the set of critical points. The polynomial expressions can be determined by linear equations. Based on these expressions, new Lasserre type semidefinite relaxations are constructed for solving polynomial optimization. We show that the hierarchy of new relaxations has finite convergence, or equivalently, the new relaxations are tight for a finite relaxation order., Comment: 27 pages

In this paper, we consider two paradigms that are developed to account for uncertainty in optimization models: robust optimization (RO) and joint estimation-optimization (JEO). We examine recent developments on efficient and scalable iterative first-order methods for these problems, and show that these iterative methods can be viewed through the lens of online convex optimization (OCO). The standard OCO framework has seen much success for its ability to handle decision-making in dynamic, uncertain, and even adversarial environments. Nevertheless, our applications of interest present further flexibility in OCO via three simple modifications to standard OCO assumptions: we introduce two new concepts of weighted regret and online saddle point problems and study the possibility of making lookahead (anticipatory) decisions. Our analyses demonstrate that these flexibilities introduced into the OCO framework have significant consequences whenever they are applicable. For example, in the strongly convex case, minimizing unweighted regret has a proven optimal bound of $$O(\mathop {\mathrm{log}}(T)/T)$$, whereas we show that a bound of O(1 / T) is possible when we consider weighted regret. Similarly, for the smooth case, considering 1-lookahead decisions results in a O(1 / T) bound, compared to $$O(1/\sqrt{T})$$ in the standard OCO setting. Consequently, these OCO tools are instrumental in exploiting structural properties of functions and results in improved convergence rates for RO and JEO. In certain cases, our results for RO and JEO match the best known or optimal rates in the corresponding problem classes without data uncertainty.

There is often a significant trade-off between formulation strength and size in mixed integer programming (MIP). When modelling convex disjunctive constraints (e.g. unions of convex sets) this trade-off can be resolved by adding auxiliary continuous variables. However, adding these variables can result in a deterioration of the computational effectiveness of the formulation. For this reason, there has been considerable interest in constructing strong formulations that do not use continuous auxiliary variables. We introduce a technique to construct formulations without these detrimental continuous auxiliary variables. To develop this technique we introduce a natural nonpolyhedral generalization of the Cayley embedding of a family of polytopes and show it inherits many geometric properties of the original embedding. We then show how the associated formulation technique can be used to construct small and strong formulation for a wide range of disjunctive constraints. In particular, we show it can recover and generalize all known strong formulations without continuous auxiliary variables., National Science Foundation (U.S.) (grant CMMI-1351619)

A local convergence result for abstract descent methods is proved. The sequence of iterates is attracted by a local (or global) minimum, stays in its neighborhood and converges within this neighborhood. This result allows algorithms to exploit local properties of the objective function. In particular, the abstract theory in this paper applies to the inertial forward--backward splitting method: iPiano---a generalization of the Heavy-ball method. Moreover, it reveals an equivalence between iPiano and inertial averaged/alternating proximal minimization and projection methods. Key for this equivalence is the attraction to a local minimum within a neighborhood and the fact that, for a prox-regular function, the gradient of the Moreau envelope is locally Lipschitz continuous and expressible in terms of the proximal mapping. In a numerical feasibility problem, the inertial alternating projection method significantly outperforms its non-inertial variants.

We consider a non-atomic congestion game on a graph, with several classes of players. Each player wants to go from his origin vertex to his destination vertex at the minimum cost and all players of a given class share the same characteristics: cost functions on each arc, and origin–destination pair. Under some mild conditions, it is known that a Nash equilibrium exists, but the computation of such an equilibrium in the multiclass case is an open problem for general functions. We consider the specific case where the cost functions are affine. We show that this problem is polynomially solvable when the number of vertices and the number of classes are fixed. In particular, it shows that the parallel two-terminal case with a fixed number of classes is polynomially solvable. On a more practical side, we propose an extension of Lemke’s algorithm able to solve this problem.

Much of statistics relies upon four key elements: a law of large numbers, a calculus to operationalize stochastic convergence, a central limit theorem, and a framework for constructing local approximations. These elements are well-understood for objects in a vector space (e.g., points or functions); however, much statistical theory does not directly translate to sets because they do not form a vector space. Building on probability theory for random sets, this paper uses variational analysis to develop operational tools for statistics with set-valued functions. These tools are first applied to nonparametric estimation (kernel regression of set-valued functions). The second application is to the problem of inverse approximate optimization, in which approximate solutions (corrupted by noise) to an optimization problem are observed and then used to estimate the amount of suboptimality of the solutions and the parameters of the optimization problem that generated the solutions. We show that previous approaches to this problem are statistically inconsistent when the data is corrupted by noise, whereas our approach is consistent under mild conditions.

We present a novel randomized block coordinate descent method for the minimization of a convex composite objective function. The method uses (approximate) partial second-order (curvature) information, so that the algorithm performance is more robust when applied to highly nonseparable or ill conditioned problems. We call the method Flexible Coordinate Descent (FCD). At each iteration of FCD, a block of coordinates is sampled randomly, a quadratic model is formed about that block and the model is minimized \emph{approximately/inexactly} to determine the search direction. An inexpensive line search is then employed to ensure a monotonic decrease in the objective function and acceptance of large step sizes. We present several high probability iteration complexity results to show that convergence of FCD is guaranteed theoretically. Finally, we present numerical results on large-scale problems to demonstrate the practical performance of the method., Comment: 31 pages, 24 figures

In Speakman and Lee (Math Oper Res 42(4):1230–1253, 2017), we analytically developed the idea of using volume as a measure for comparing relaxations in the context of spatial branch-and-bound. Specifically, for trilinear monomials, we analytically compared the three possible “double-McCormick relaxations” with the tight convex-hull relaxation. Here, again using volume as a measure, for the convex-hull relaxation of trilinear monomials, we establish simple rules for determining the optimal branching variable and optimal branching point. Additionally, we compare our results with current software practice.

We consider general nonlinear programming problems with cardinality constraints. By relaxing the binary variables which appear in the natural mixed-integer programming formulation, we obtain an almost equivalent nonlinear programming problem, which is thus still difficult to solve. Therefore, we apply a Scholtes-type regularization method to obtain a sequence of easier to solve problems and investigate the convergence of the obtained KKT points. We show that such a sequence converges to an S-stationary point, which corresponds to a local minimizer of the original problem under the assumption of convexity. Additionally, we consider portfolio optimization problems where we minimize a risk measure under a cardinality constraint on the portfolio. Various risk measures are considered, in particular Value-at-Risk and Conditional Value-at-Risk under normal distribution of returns and their robust counterparts under moment conditions. For these investment problems formulated as nonlinear programming problems with cardinality constraints we perform a numerical study on a large number of simulated instances taken from the literature and illuminate the computational performance of the Scholtes-type regularization method in comparison to other considered solution approaches: a mixed-integer solver, a direct continuous reformulation solver and the Kanzow-Schwartz regularization method, which has already been applied to Markowitz portfolio problems., 23 pages, 4 figures

Pairwise comparison matrix as a crucial component of AHP, presents the prefer- ence relations among alternatives. However, in many cases, the pairwise comparison matrix is difficult to complete, which obstructs the subsequent operations of the clas- sical AHP. In this paper, based on DEMATEL which has ability to derive the total relation matrix from direct relation matrix, a new completion method for incomplete pairwise comparison matrix is proposed. The proposed method provides a new per- spective to estimate the missing values with explicit physical meaning. Besides, the proposed method has low computational cost. This promising method has a wide application in multi-criteria decision-making., Comment: 29 pages

Maximizing the sum of two generalized Rayleigh quotients (SRQ) can be reformulated as a one-dimensional optimization problem, where the function value evaluations are reduced to solving semi-definite programming (SDP) subproblems. In this paper, we first use the dual SDP subproblem to construct an explicit overestimation and then propose a branch-and-bound algorithm to globally solve (SRQ). Numerical results demonstrate that it is even more efficient than the recent SDP-based heuristic algorithm., 12pages

In this paper, we present a conditional gradient type (CGT) method for solving a class of composite optimization problems where the objective function consists of a (weakly) smooth term and a (strongly) convex regularization term. While including a strongly convex term in the subproblems of the classical conditional gradient (CG) method improves its rate of convergence, it does not cost per iteration as much as general proximal type algorithms. More specifically, we present a unified analysis for the CGT method in the sense that it achieves the best-known rate of convergence when the weakly smooth term is nonconvex and possesses (nearly) optimal complexity if it turns out to be convex. While implementation of the CGT method requires explicitly estimating problem parameters like the level of smoothness of the first term in the objective function, we also present a few variants of this method which relax such estimation. Unlike general proximal type parameter free methods, these variants of the CGT method do not require any additional effort for computing (sub)gradients of the objective function and/or solving extra subproblems at each iteration. We then generalize these methods under stochastic setting and present a few new complexity results. To the best of our knowledge, this is the first time that such complexity results are presented for solving stochastic weakly smooth nonconvex and (strongly) convex optimization problems.

The second-order cone plays an important role in convex optimization and has strong expressive abilities despite its apparent simplicity. Second-order cone formulations can also be solved more efficiently than semidefinite programming in general. We consider the following question, posed by Lewis and Glineur, Parrilo, Saunderson: is it possible to express the general positive semidefinite cone using second-order cones? We provide a negative answer to this question and show that the 3x3 positive semidefinite cone does not admit any second-order cone representation. Our proof relies on exhibiting a sequence of submatrices of the slack matrix of the 3x3 positive semidefinite cone whose "second-order cone rank" grows to infinity. We also discuss the possibility of representing certain slices of the 3x3 positive semidefinite cone using the second-order cone., Comment: 8 pages

The standard assumption for proving linear convergence of first order methods for smooth convex optimization is the strong convexity of the objective function, an assumption which does not hold for many practical applications. In this paper, we derive linear convergence rates of several first order methods for solving smooth non-strongly convex constrained optimization problems, i.e. involving an objective function with a Lipschitz continuous gradient that satisfies some relaxed strong convexity condition. In particular, in the case of smooth constrained convex optimization, we provide several relaxations of the strong convexity conditions and prove that they are sufficient for getting linear convergence for several first order methods such as projected gradient, fast gradient and feasible descent methods. We also provide examples of functional classes that satisfy our proposed relaxations of strong convexity conditions. Finally, we show that the proposed relaxed strong convexity conditions cover important applications ranging from solving linear systems, Linear Programming, and dual formulations of linearly constrained convex problems., Comment: 36 pages, 4 figures

The randomized version of the Kaczmarz method for the solution of consistent linear systems is known to converge linearly in expectation. And even in the possibly inconsistent case, when only noisy data is given, the iterates are expected to reach an error threshold in the order of the noise-level with the same rate as in the noiseless case. In this work we show that the same also holds for the iterates of the recently proposed randomized sparse Kaczmarz method for recovery of sparse solutions. Furthermore we consider the more general setting of convex feasibility problems and their solution by the method of randomized Bregman projections. This is motivated by the observation that, similarly to the Kaczmarz method, the Sparse Kaczmarz method can also be interpreted as an iterative Bregman projection method to solve a convex feasibility problem. We obtain expected sublinear rates for Bregman projections with respect to a general strongly convex function. Moreover, even linear rates are expected for Bregman projections with respect to smooth or piecewise linear-quadratic functions, and also the regularized nuclear norm, which is used in the area of low rank matrix problems.

In the article we present a general theory of augmented Lagrangian functions for cone constrained optimization problems that allows one to study almost all known augmented Lagrangians for cone constrained programs within a unified framework. We develop a new general method for proving the existence of global saddle points of augmented Lagrangian functions, called the localization principle. The localization principle unifies, generalizes and sharpens most of the known results on existence of global saddle points, and, in essence, reduces the problem of the existence of saddle points to a local analysis of optimality conditions. With the use of the localization principle we obtain first necessary and sufficient conditions for the existence of a global saddle point of an augmented Lagrangian for cone constrained minimax problems via both second and first order optimality conditions. In the second part of the paper, we present a general approach to the construction of globally exact augmented Lagrangian functions. The general approach developed in this paper allowed us not only to sharpen most of the existing results on globally exact augmented Lagrangians, but also to construct first globally exact augmented Lagrangian functions for equality constrained optimization problems, for nonlinear second order cone programs and for nonlinear semidefinite programs. These globally exact augmented Lagrangians can be utilized in order to design new superlinearly (or even quadratically) convergent optimization methods for cone constrained optimization problems., This is a preprint of an article published by Springer in Journal of Global Optimization (2018). The final authenticated version is available online at: http://dx.doi.org/10.1007/s10898-017-0603-0

The subgradient extragradient method for solving the variational inequality (VI) problem, which is introduced by Censor et al. (J. Optim. Theory Appl. 148, 318–335, 2011), replaces the second projection onto the feasible set of the VI, in the extragradient method, with a subgradient projection onto some constructible half-space. Since the method has been introduced, many authors proposed extensions and modifications with applications to various problems. In this paper, we introduce a modified subgradient extragradient method by improving the stepsize of its second step. Convergence of the proposed method is proved under standard and mild conditions and primary numerical experiments illustrate the performance and advantage of this new subgradient extragradient variant.

In this paper, an upper semismooth function is defined to be a lower semicontinuous function whose radial subderivative satisfies a mild directional upper semicontinuity property. Examples of upper semismooth functions are the proper lower semicontinuous convex functions, the lower-C$^1$ functions, the regular directionally Lipschitz functions, the Mifflin semismooth functions, the Thibault-Zagrodny directionally stable functions. It is shown that the radial subderivative of such functions can be recovered from any subdifferential of the function. It is also shown that these functions are subdifferentially determined, in the sense that if two functions have the same subdifferential and one of the functions is upper semismooth, then the two functions are equal up to an additive constant., 16 pages

This paper studies the expected optimal value of a mixed 0-1 programming problem with uncertain objective coefficients following a joint distribution. We assume that the true distribution is not known exactly, but a set of independent samples can be observed. Using the Wasserstein metric, we construct an ambiguity set centered at the empirical distribution from the observed samples and containing the true distribution with a high statistical guarantee. The problem of interest is to investigate the bound on the expected optimal value over the Wasserstein ambiguity set. Under standard assumptions, we reformulate the problem into a copositive program, which naturally leads to a tractable semidefinite-based approximation. We compare our approach with a moment-based approach from the literature on three applications. Numerical results illustrate the effectiveness of our approach., Comment: 29 pages, 8 figures, and 3 tables

We study two-stage adjustable robust linear programming in which the right-hand sides are uncertain and belong to a convex, compact uncertainty set. This problem is NP-hard, and the affine policy is a popular, tractable approximation. We prove that under standard and simple conditions, the two-stage problem can be reformulated as a copositive optimization problem, which in turn leads to a class of tractable, semidefinite-based approximations that are at least as strong as the affine policy. We investigate several examples from the literature demonstrating that our tractable approximations significantly improve the affine policy. In particular, our approach solves exactly in polynomial time a class of instances of increasing size for which the affine policy admits an arbitrarily large gap., Comment: 30 pages and 2 tables