Search

In this paper, an augmented memoryless BFGS quasi-Newton method was proposed for solving unconstrained optimization problems. Based on a new modified secant equation, an augmented memoryless BFGS update formula and an efficient optimization algorithm were established. To improve the stability of the numerical experiment, we obtained the scaling parameter by minimizing the upper bound of the condition number. The global convergence of the algorithm was proved, and numerical experiments showed that the algorithm was efficient. [ABSTRACT FROM AUTHOR]

Based on the memoryless BFGS (Broyden–Fletcher–Goldfarb–Shanno) updating formula of a recent well-structured diagonal approximation of the Hessian, we propose an improved proximal method for solving the minimization problem of nonsmooth composite functions. More exactly, a diagonally scaled matrix is iteratively used to approximate Hessian of the smooth ingredient of the cost function, which leads to straightly determining the search directions in each iteration. Afterward, in light of the Zhang–Hager nonmonotone scheme, a nonmonotone technique for performing the line search for the unconstrained optimization models with composite cost functions is devised. What is more, we address convergence of the suggested proximal algorithm. We close the discussion by empirically studying performance of the proposed algorithm on some large-scale compressive sensing and sparse logistic regression problems. [ABSTRACT FROM AUTHOR]

In this paper, we address the preconditioned iterative solution of the saddle-point linear systems arising from the (regularized) Interior Point method applied to linear and quadratic convex programming problems, typically of large scale. Starting from the well-studied Constraint Preconditioner, we review a number of inexact variants with the aim to reduce the computational cost of the preconditioner application within the Krylov subspace solver of choice. In all cases we illustrate a spectral analysis showing the conditions under which a good clustering of the eigenvalues of the preconditioned matrix can be obtained, which foreshadows (at least in case PCG/MINRES Krylov solvers are used), a fast convergence of the iterative method. Results on a set of large size optimization problems confirm that the Inexact variants of the Constraint Preconditioner can yield efficient solution methods. [ABSTRACT FROM AUTHOR]

Spectral conjugate gradient methods are considered as an efficient family of conjugate gradient methods to solve unconstrained optimization problems. In this work, based on the scaled memoryless BFGS update, a spectral Polak–Ribière–Polyak conjugate gradient method is proposed. It is established that the method ensures the sufficient descent property independent of line search techniques as well as convexity of the objective function. The global convergence of the method is shown with the strong Wolfe line search strategy for general functions. Finally, the practical competencies of the method are investigated by numerical experiments on a set of test functions of CUTEr library. The results demonstrate computational efficiency of the proposed method. [ABSTRACT FROM AUTHOR]

• A rank-one modification is performed on the classical BFGS update for the Hessian. • The given updating formula satisfies an extended secant equation. • An augmented version of the BFGS method is developed. • The method can be efficiently used to solve practical large-scale optimization problems. Making a rank-one modification on the classical BFGS (Broyden–Fletcher–Goldfarb–Shanno) updating formula, we develop a class of augmented BFGS methods. The suggested formula can be considered as a hybridization of the basic BFGS updating formula for Hessian with an additional rank-one term embedded to guarantee a general modified secant equation. By using the well-known Sherman–Morrison formula, the inverse of a memoryless version of the given updating formula is computed to be applied for solving large-scale problems. Convergence analysis is concisely carried out as well. At last, the practical merits of the method are investigated by numerical tests on a set of CUTEr problems as well as the well-known compressed sensing problem. Results show the computational efficiency of the given method. [ABSTRACT FROM AUTHOR]

In this article, we address the efficient numerical solution of linear and quadratic programming problems, often of large scale. With this aim, we devise an infeasible interior point method, blended with the proximal method of multipliers, which in turn results in a primal‐dual regularized interior point method. Application of this method gives rise to a sequence of increasingly ill‐conditioned linear systems which cannot always be solved by factorization methods, due to memory and CPU time restrictions. We propose a novel preconditioning strategy which is based on a suitable sparsification of the normal equations matrix in the linear case, and also constitutes the foundation of a block‐diagonal preconditioner to accelerate MINRES for linear systems arising from the solution of general quadratic programming problems. Numerical results for a range of test problems demonstrate the robustness of the proposed preconditioning strategy, together with its ability to solve linear systems of very large dimension. [ABSTRACT FROM AUTHOR]

Copyright of Sultan Qaboos University Journal for Science is the property of Sultan Qaboos University and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

The alternating direction method of multipliers (ADMM) is an effective method for solving convex problems from a wide range of fields. At each iteration, the classical ADMM solves two subproblems exactly. However, in many applications, it is expensive or impossible to obtain the exact solutions of the subproblems. To overcome the difficulty, some proximal terms are added to the subproblems. This class of methods typically solves the original subproblem approximately and hence requires more iterations. This fact urges us to consider that a special proximal term can yield better results than the classical ADMM. In this paper, we propose a proximal ADMM whose regularization matrix in the proximal term is generated by the BFGS update (or limited memory BFGS) at every iteration. These types of matrices use second-order information of the objective function. The convergence of the proposed method is proved under certain assumptions. Numerical results are presented to demonstrate the effectiveness of the proposed proximal ADMM. [ABSTRACT FROM AUTHOR]

Copyright of Baghdad Science Journal is the property of Republic of Iraq Ministry of Higher Education & Scientific Research (MOHESR) and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Abstract In this paper, a modified BFGS algorithm is proposed for unconstrained optimization. The proposed algorithm has the following properties: (i) a nonmonotone line search technique is used to obtain the step size α k $\alpha_{k}$ to improve the effectiveness of the algorithm; (ii) the algorithm possesses not only global convergence but also superlinear convergence for generally convex functions; (iii) the algorithm produces better numerical results than those of the normal BFGS method.

A block version of the Broyden-Fletcher-Goldfarb-Shanno (BFGS) variable metric update formula and its modifications are investigated. In spite of the fact that this formula satisfies the quasi-Newton conditions with all used difference vectors and that the improvement of convergence is the best one in some sense for quadratic objective functions, for general functions, it does not guarantee that the corresponding direction vectors are descent directions. To overcome this difficulty, but at the same time utilize the advantageous properties of the block BFGS update, a block version of the limited-memory variable metric BNS method for large-scale unconstrained optimization is proposed. The global convergence of the algorithm is established for convex sufficiently smooth functions. Numerical experiments demonstrate the efficiency of the new method. [ABSTRACT FROM AUTHOR]

In this paper, we present two conjugate gradient methods choices for solving unconstrained optimization problems. This attempts is to find suitable choices for parameter of a nonlinear conjugate gradient method proposed by Dai and Liao based on the matrix analysis and using the memoryless BFGS updating formula. Numerical results show that the proposed method is efficient for the unconstrained problems in the CUTEr collection. [ABSTRACT FROM AUTHOR]

In this paper, a modified BFGS algorithm is proposed for unconstrained optimization. The proposed algorithm has the following properties: (i) a nonmonotone line search technique is used to obtain the step size $\alpha_{k}$ to improve the effectiveness of the algorithm; (ii) the algorithm possesses not only global convergence but also superlinear convergence for generally convex functions; (iii) the algorithm produces better numerical results than those of the normal BFGS method. [ABSTRACT FROM AUTHOR]

In this paper, we present preconditioning techniques to accelerate the convergence of Krylov solvers at each step of an Inexact Newton's method for the computation of the leftmost eigenpairs of large and sparse symmetric positive definite matrices arising in large-scale scientific computations. We propose a two-stage spectral preconditioning strategy: The first stage produces a very rough approximation of a number of the leftmost eigenvectors. The second stage uses these approximations as starting vectors and also to construct the tuned preconditioner from an initial inverse approximation of the coefficient matrix, as proposed by Martínez. In the framework of the Implicitly Restarted Lanczos method. The action of this spectral preconditioner results in clustering a number of the eigenvalues of the preconditioned matrices close to one. We also study the combination of this approach with a BFGS-style updating of the proposed spectral preconditioner as described by Bergamaschi and Martínez. Extensive numerical testing on a set of representative large SPD matrices gives evidence of the acceleration provided by these spectral preconditioners. [ABSTRACT FROM AUTHOR]

Hybridizing the three-term conjugate gradient method proposed by Zhang et al. and the nonlinear conjugate gradient method proposed by Dai and Liao based on the scaled memoryless BFGS update, a one-parameter class of four-term conjugate gradient methods is proposed. It is shown that the suggested class of conjugate gradient methods possesses the suficient descent property, without convexity assumption on the objective function. A brief global convergence analysis is made for uniformly convex objective functions. Results of numerical comparisons are reported. They demonstrate eficiency of a method of the proposed class in the sense of the Dolan-Moré performance profile. [ABSTRACT FROM AUTHOR]

In this paper, we have used one of the preconditioned conjugate gradient algorithm with the Quasi – Newton approximation; namely the BFGS preconditioned algorithm which was suggested by (AL-Bayati and Aref, 2001). In this paper we have suggested a new algorithm for constrained optimization with robust numerical results, for solving constrained optimization problems.

In this paper, we study a class of tuned preconditioners that will be designed to accelerate both the DACG-Newton method and the implicitly restarted Lanczos method for the computation of the leftmost eigenpairs of large and sparse symmetric positive definite matrices arising in large-scale scientific computations. These tuning strategies are based on low-rank modifications of a given initial preconditioner. We present some theoretical properties of the preconditioned matrix. We experimentally show how the aforementioned methods benefit from the acceleration provided by these tuned/deflated preconditioners. Comparisons are carried out with the Jacobi-Davidson method onto matrices arising from various large realistic problems arising from finite element discretization of PDEs modeling either groundwater flow in porous media or geomechanical processes in reservoirs. The numerical results show that the Newton-based methods (which includes also the Jacobi-Davidson method) are to be preferred to the - yet efficiently implemented - implicitly restarted Lanczos method whenever a small to moderate number of eigenpairs is required. Copyright © 2016 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

In this paper, we consider a trust-region method for solving nonlinear equations which employs a new nonmonotone technique. A strong nonmonotone strategy and a weaker nonmonotone strategy can be obtained by choosing the parameter adaptively. Thus, the disadvantages of the traditional nonmonotone strategy can be avoided. It does not need to compute the Jacobian matrix at every iteration, so that the workload and time are decreased. Theoretical analysis indicates that the new algorithm preserves the global convergence under classical assumptions. Moreover, superlinear and quadratic convergence are established under suitable conditions. Numerical experiments show the efficiency and effectiveness of the proposed method for solving nonlinear equations. [ABSTRACT FROM PUBLISHER]

In this paper, we propose an efficiently preconditioned Newton method for the computation of the leftmost eigenpairs of large and sparse symmetric positive definite matrices. A sequence of preconditioners based on the BFGS update formula is proposed, for the preconditioned conjugate gradient solution of the linearized Newton system to solveAu=q(u)u,q(u) being the Rayleigh quotient. We give theoretical evidence that the sequence of preconditioned Jacobians remains close to the identity matrix if the initial preconditioned Jacobian is so. Numerical results onto matrices arising from various realistic problems with size up to one million unknowns account for the efficiency of the proposed algorithm which reveals competitive with the Jacobi–Davidson method on all the test problems. [ABSTRACT FROM AUTHOR]

Following the scaled conjugate gradient methods proposed by Andrei, we hybridize the memoryless BFGS preconditioned conjugate gradient method suggested by Shanno and the spectral conjugate gradient method suggested by Birgin and Martínez based on a modified secant equation suggested by Yuan, and propose two modified scaled conjugate gradient methods. The interesting features of our methods are applying the function values in addition to the gradient values and satisfying the sufficient descent condition for the generated search directions which leads to the global convergence for uniformly convex functions. Numerical comparisons between the implementations of one of our methods which generates descent search directions for general functions and an efficient scaled conjugate gradient method proposed by Andrei are made on a set of unconstrained optimization test problems from the CUTEr collection, using the performance profile introduced by Dolan and Moré. [ABSTRACT FROM AUTHOR]

By making a convex combination of the modified secant equations proposed by Yuan and Wei et al., a hybrid secant equation and also, a modified BFGS algorithm is proposed. The hybridization parameter is effectively computed using the available information of recent iterations. Under proper conditions, it is shown that the proposed algorithm is globally, locally and superlinearly convergent. By using the performance profile introduced by Dolan and Moré, a comparison between the implementations of the proposed algorithm and two efficient modified BFGS algorithms proposed by Yuan and Wei et al., on a set of unconstrained optimization test problems from the CUTEr collection, is done. Numerical results demonstrating the efficiency of the proposed modified BFGS algorithm are reported. [ABSTRACT FROM AUTHOR]

In this paper, a new trust-region subproblem combining with the BFGS update is proposed for solving nonlinear equations, where the trust region radius is defined by a new way. The global convergence without the nondegeneracy assumption and the quadratic convergence are obtained under suitable conditions. Numerical results show that this method is more effective than the norm method. [ABSTRACT FROM AUTHOR]

Abstract: Filter methods were initially designed for nonlinear programming problems by Fletcher and Leyffer. In this paper we propose a secant algorithm with line search filter method for nonlinear equality constrained optimization. The algorithm yields the global convergence under some reasonable conditions. By using the Lagrangian function value in the filter we establish that the proposed algorithm can overcome the Maratos effect without using second order correction step, so that fast local superlinear convergence to second order sufficient local solution is achieved. The primary numerical results are presented to confirm the robustness and efficiency of our approach. [ABSTRACT FROM AUTHOR]

Abstract: In this paper, we propose a BFGS trust-region method for solving symmetric nonlinear equations. The global convergence and the superlinear convergence of the presented method will be established under favorable conditions. Numerical results show that the new algorithm is effective. [Copyright &y& Elsevier]

The Broyden–Fletcher–Goldfarb–Shanno (BFGS) method plays an important role among the quasi-Newton algorithms for nonconvex and unconstrained optimization problems. However, in the proof of global convergence, BFGS-type methods generally need to assume that the gradient of the objective function is Lipschitz continuous. This issue prompts us to try to find quasi-Newton method for gradient non-Lipschitz continuous and nonconvex optimization based on the classical BFGS formula. In this paper, we propose an adaptive scaling damped BFGS method for gradient non-Lipschitz continuous and nonconvex problems. With Armijo or Weak Wolfe–Powell (WWP) line search, global convergence can be obtained. Under suitable conditions the convergence rate is superlinear with WWP-type line search. Applications of the given algorithms include the tested optimization problems, which turn out the proposed method is powerful and promising. • We develop a type of damped BFGS methods that solves nonconvex problems. • The global convergence is established without gradient Lipschitz continuity. • In the case of weaker assumptions, correct the structure of the eigenvalues. • The method convergence to a stationary point under Armijo or WWP line search. • The convergence rate is superlinear under suitable conditions. [ABSTRACT FROM AUTHOR]

In a numerical study, Contreras and Tapia demonstrated the value of selectively sizing the BFGS secant update. A conclusion of the current study is that there is value in following a sizing of the BFGS update with a shifting of the BFGS update. Our motivation for such a strategy comes from the known fact that the BFGS update is prone to producing updates with large eigenvalues. Hence, sizing may contribute to near singularity, and shifting can help compensate for this deficiency. We follow Contreras and Tapia in our form of selective sizing. Our form of shifting can be viewed as switching from the BFGS update to an alternative member of the Broyden family of updates. Our shift is obtained by minimizing a weighted form of the Byrd-Nocedal measure over a sized Broyden family of secant updates. Numerically, our most effective shift is the one that gives the member of the class which is closest to steepest descent in the Byrd-Nocedal measure. [ABSTRACT FROM AUTHOR]

This paper analyzes rank modification of symmetric positive definite matrices H of the form H - M + P, where H - M denotes a step of reducing H to a lower-rank, symmetric and positive semidefinite matrix and (H - M) + P denotes a step of restoring H - M to a sym- metric positive definite matrix. These steps and their generalizations for rectangular matrices are fully characterized. The well-known BFGS and DFP updates used in Hessian and inverse Hessian approximations provided the motivation for the generalizations and are special cases with H and P having rank one. [ABSTRACT FROM AUTHOR]

We describe a variational principle based upon minimizing the extent to which the inverse hessian approximation, say H, violates the quasi-Newton relation, on the step immediately prior to the step used to construct H. Its application to the case when line searches are exact suggests use of the BFGS update. [ABSTRACT FROM AUTHOR]