Search

We introduce a twice differentiable augmented Lagrangian for nonlinear optimization with general inequality constraints and show that a strict local minimizer of the original problem is an approximate strict local solution of the augmented Lagrangian. A novel augmented Lagrangian method of multipliers (ALM) is then presented. Our method is originated from a generalization of the Hetenes-Powell augmented Lagrangian, and is a combination of the augmented Lagrangian and the interior-point technique. It shares a similar algorithmic framework with existing ALMs for optimization with inequality constraints, but it can use the second derivatives and does not depend on projections on the set of inequality constraints. In each iteration, our method solves a twice continuously differentiable unconstrained optimization subproblem on primal variables. The dual iterates, penalty and smoothing parameters are updated adaptively. The global and local convergence are analyzed. Without assuming any constraint qualification, it is proved that the proposed method has strong global convergence. The method may converge to either a Kurash-Kuhn-Tucker (KKT) point or a singular stationary point when the converging point is a minimizer. It may also converge to an infeasible stationary point of nonlinear program when the problem is infeasible. Furthermore, our method is capable of rapidly detecting the possible infeasibility of the solved problem. Under suitable conditions, it is locally linearly convergent to the KKT point, which is consistent with ALMs for optimization with equality constraints. The preliminary numerical experiments on some small benchmark test problems demonstrate our theoretical results., Comment: 31 pages

Based on solving an equivalent parametric equality constrained mini-max problem of the classic logarithmic-barrier subproblem, we present a novel primal-dual interior-point relaxation method for nonlinear programs with general equality and nonnegative constraints. In each iteration, our method approximately solves the KKT system of a parametric equality constrained mini-max subproblem, which avoids the requirement that any primal or dual iterate is an interior-point. The method has some similarities to the warmstarting interior-point methods in relaxing the interior-point requirement and is easily extended for solving problems with general inequality constraints. In particular, it has the potential to circumvent the jamming difficulty that appears with many interior-point methods for nonlinear programs and improve the ill conditioning of existing primal-dual interior-point methods as the barrier parameter is small. A new smoothing approach is introduced to develop our relaxation method and promote convergence of the method. Under suitable conditions, it is proved that our method can be globally convergent and locally quadratically convergent to the KKT point of the original problem. The preliminary numerical results on a well-posed problem for which many interior-point methods fail to find the minimizer and a set of test problems from the CUTEr collection show that our method is efficient., Comment: accepted by Mathematical Methods of Operations Research, access to a view-only version https://rdcu.be/cUFBe

We present a primal-dual majorization-minimization method for solving large-scale linear programs. A smooth barrier augmented Lagrangian (SBAL) function with strict convexity for the dual linear program is derived. The majorization-minimization approach is naturally introduced to develop the smoothness and convexity of the SBAL function. Our method only depends on a factorization of the constant matrix independent of iterations and does not need any computation on step sizes, thus can be expected to be particularly appropriate for large-scale linear programs. The method shares some similar properties to the first-order methods for linear programs, but its convergence analysis is established on the differentiability and convexity of our SBAL function. The global convergence is analyzed without prior requiring either the primal or dual linear program to be feasible. Under the regular conditions, our method is proved to be globally linearly convergent, and a new iteration complexity result is given., 25 pages

Two dimensional quadratic termination property has shown potential benefits of improving performance of gradient methods. However, higher dimensional quadratic termination property is rarely investigated in the literature. We introduce a scheme for achieving three dimensional quadratic termination of the gradient method. Based on the scheme, a novel stepsize is derived which can be incorporated with some well known gradient methods such as the steepest descent (SD) and Barzilai--Borwein (BB) methods to obtain three dimensional quadratic termination. A remarkable feature of the novel stepsize is that its computation does not require additional matrix-vector products or vector inner products. By incorporating the new stepsize with SD steps and BB steps, respectively, we develop efficient gradient methods for quadratic optimization. We establish $R$-linear convergence of the proposed methods. Numerical results show that the three dimensional quadratic termination property can significantly improve performance of the associated gradient method., Comment: 23 pages

The Barzilai-Borwein (BB) gradient method is efficient for solving large-scale unconstrained problems to the modest accuracy and has a great advantage of being easily extended to solve a wide class of constrained optimization problems. In this paper, we propose a new stepsize to accelerate the BB method by requiring finite termination for minimizing two-dimensional strongly convex quadratic function. Combing with this new stepsize, we develop gradient methods which adaptively take the nonmonotone BB stepsizes and certain monotone stepsizes for minimizing general strongly convex quadratic function. Furthermore, by incorporating nonmonotone line searches and gradient projection techniques, we extend these new gradient methods to solve general smooth unconstrained and bound constrained optimization. Extensive numerical experiments show that our strategies of properly inserting monotone gradient steps into the nonmonotone BB method could significantly improve its performance and the new resulted methods can outperform the most successful gradient decent methods developed in the recent literature., Comment: 26 pages, 5 figures

A novel gradient stepsize is derived at the motivation of equipping the Barzilai-Borwein (BB) method with two dimensional quadratic termination property. A remarkable feature of the novel stepsize is that its computation only depends on the BB stepsizes in previous iterations and does not require any exact line search or the Hessian, and hence it can easily be extended for nonlinear optimization. By adaptively taking long BB steps and some short steps associated with the new stepsize, we develop an efficient gradient method for quadratic optimization and general unconstrained optimization and extend it to solve extreme eigenvalues problems. The proposed method is further extended for box-constrained optimization and singly linearly box-constrained optimization by incorporating gradient projection techniques. Numerical experiments demonstrate that the proposed method outperforms the most successful gradient methods in the literature., Comment: 25 pages, 4 figures

We prove that the classic logarithmic barrier problem is equivalent to a particular logarithmic barrier positive relaxation problem with barrier and scaling parameters. Based on the equivalence, a line-search primal-dual interior-point relaxation method for nonlinear programs is presented. Our method does not require any primal or dual iterates to be interior-points, which is prominently different from the existing interior-point methods in the literature. A new logarithmic barrier penalty function dependent on both primal and dual variables is used to prompt the global convergence of the method, where the penalty parameter is updated adaptively. Without assuming any regularity condition, it is proved that our method will terminate at an approximate KKT point of the original problem provided the barrier parameter tends zero. Otherwise, either an approximate infeasible stationary point or an approximate singular stationary point of the original problem will be found. Some preliminary numerical results are reported, including the results for a well-posed problem for which many line-search interior-point methods were demonstrated not to be globally convergent, a feasible problem for which the LICQ and the MFCQ fail to hold at the solution and an infeasible problem, and for some standard test problems of the CUTE collection. These results show that our algorithm is not only efficient for well-posed feasible problems, but also is applicable for some ill-posed feasible problems and some even infeasible problems., Comment: 25 pages