Search

In this work, we propose nonlinear conjugate gradient methods for finding critical points of vector-valued functions with respect to the partial order induced by a closed, convex, and pointed cone with nonempty interior. No convexity assumption is made on the objectives. The concepts of Wolfe and Zoutendjik conditions are extended for the vector-valued optimization. In particular, we show that there exist intervals of step sizes satisfying the Wolfe-type conditions. The convergence analysis covers the vector extensions of the Fletcher{Reeves, conjugate descent, Dai-Yuan, Polak-Ribière-Polyak, and Hestenes-Stiefel parameters that retrieve the classical ones in the scalar minimization case. Under inexact line searches and without regular restarts, we prove that the sequences generated by the proposed methods find points that satisfy the first-order necessary condition for Pareto-optimality. Numerical experiments illustrating the practical behavior of the methods are presented. [ABSTRACT FROM AUTHOR]

Conjugate gradient methods are an important class of methods for unconstrained optimization, especially for large-scale problems. Recently, they have been much studied. In this paper, a new family of conjugate gradient method is proposed for unconstrained optimization. This method includes the already existing two practical nonlinear conjugate gradient methods, which produces a descent search direction at every iteration and converges globally provided that the line search satisfies the Wolfe conditions. The numerical experiments are done to test the efficiency of the new method, which implies the new method is promising. In addition the methods related to this family are uniformly discussed. [ABSTRACT FROM AUTHOR]

In this paper, we propose a new conjugate gradient (CG) method which belongs to the CG methods of Dai–Liao family [New conjugacy conditions and related nonlinear conjugate gradient methods, Appl. Math. Optim. 43 (2001), pp. 87–101]. Babaie-Kafaki et al. [Two new conjugate gradient methods based on modified secant equations, J. Comput. Appl. Math. 234 (2010), pp. 1374–1386] made some modifications on the Yabe and Takano's CG approach [Global convergence properties of nonlinear conjugate gradient methods with modified secant condition, Comput. Optim. Appl. 28 (2004), pp. 203–225] and received some appealing results in theory and practice. Here, we introduce an efficient updating rule for the parameters of the Yabe and Takano's CG algorithm. Under some standard assumptions, we establish the global convergence property of the new suggested algorithm on uniformly convex and general functions. Numerical results on some testing problems from CUTEr collection show the priority of the proposed method to some existing CG methods in practice. [ABSTRACT FROM AUTHOR]

In this paper, a modified Polak–Ribi‘ere–Polyak (MPRP) conjugate gradient method for smooth unconstrained optimization is proposed. This method produces at each iteration a descent direction, and this property is independent of the line search adopted. Under standard assumptions, we prove that the MPRP method using strong Wolfe conditions is globally convergent. The results of computational experiments are reported and show the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Abstract: Conjugate gradient methods have played a special role for solving large scale optimization problems due to the simplicity of their iteration, convergence properties and their low memory requirements. In this work, we propose a new class of spectral conjugate gradient methods which ensures sufficient descent independent of the accuracy of the line search. Moreover, an attractive property of our proposed methods is that they achieve a high-order accuracy in approximating the second order curvature information of the objective function by utilizing the modified secant condition proposed by Babaie-Kafaki et al. [S. Babaie-Kafaki, R. Ghanbari, N. Mahdavi-Amiri, Two new conjugate gradient methods based on modified secant equations, Journal of Computational and Applied Mathematics 234 (2010) 1374–1386]. Further, a global convergence result for general functions is established provided that the line search satisfies the Wolfe conditions. Our numerical experiments indicate that our proposed methods are preferable and in general superior to the classical conjugate gradient methods in terms of efficiency and robustness. [Copyright &y& Elsevier]

Abstract: In this paper, we consider the Polak–Ribière (or Polak–Ribière plus) conjugate gradient method for solving optimality condition of an unconstrained minimization problem. We give two new steplength rules only using gradient, and under gradient-Lipschitz assumption prove this method’s global convergence correspondingly. Then, we develop a practical Polak–Ribière plus method whose steplength is located by one inequality only using gradient, and report promising numerical results on high accuracy solution for some standard test problems when compared to the state-of-art methods in this research direction. Importantly, our work provides a new idea of devising a practical version of the celebrated Polak–Ribière (or Polak–Ribière plus) method. [Copyright &y& Elsevier]

Abstract: Conjugate gradient methods have been paid attention to, because they can be directly applied to large-scale unconstrained optimization problems. In order to incorporate second order information of the objective function into conjugate gradient methods, Dai and Liao (2001) proposed a conjugate gradient method based on the secant condition. However, their method does not necessarily generate a descent search direction. On the other hand, Hager and Zhang (2005) proposed another conjugate gradient method which always generates a descent search direction. In this paper, combining Dai–Liao’s idea and Hager–Zhang’s idea, we propose conjugate gradient methods based on secant conditions that generate descent search directions. In addition, we prove global convergence properties of the proposed methods. Finally, preliminary numerical results are given. [Copyright &y& Elsevier]

Abstract: Simple modifications of the limited-memory BFGS method (L-BFGS) for large scale unconstrained optimization are considered, which consist in corrections (derived from the idea of conjugate directions) of the used difference vectors, utilizing information from the preceding iteration. For quadratic objective functions, the improvement of convergence is the best one in some sense and all stored difference vectors are conjugate for unit stepsizes. Global convergence of the algorithm is established for convex sufficiently smooth functions. Numerical experiments indicate that the new method often improves the L-BFGS method significantly. [Copyright &y& Elsevier]

Abstract: Nonlinear conjugate gradient (CG) methods have played an important role in solving large-scale unconstrained optimization. Their wide application in many fields is due to their low memory requirements and global convergence properties. Numerous studies and modifications have been conducted recently to improve this method. In this paper, a new class of conjugate gradient coefficients (βk ) that possess global convergence properties is presented. The global convergence result is established using exact line searches. Numerical result shows that the proposed formula is superior and more efficient when compared to other CG coefficients. [Copyright &y& Elsevier]

For solving large-scale unconstrained minimization problems, the nonlinear conjugate gradient method is welcome due to its simplicity, low storage, efficiency and nice convergence properties. Among all the methods in the framework, the conjugate gradient descent algorithm — CG_DESCENT is very popular, in which the generated directions descend automatically, and this nice property is independent of any line search used. In this paper, we generalize CG_DESCENT with two Barzilai–Borwein steplength reused cyclically. We show that the resulting algorithm owns attractive sufficient descent property and converges globally under some mild conditions. We test the proposed algorithm by using a large set of unconstrained problems with high dimensions in CUTEr library. The numerical comparisons with the state-of-the-art algorithm CG_DESCENT illustrate that the proposed method is effective, competitive, and promising. [ABSTRACT FROM AUTHOR]

Sufficient descent condition is very crucial in establishing the global convergence of nonlinear conjugate gradient method. In this paper, we modified two conjugate gradient methods such that both methods satisfy this property. Under suitable conditions, we prove the global convergence of the proposed methods. Numerical results show that the proposed methods are efficient for the given test problems. [ABSTRACT FROM PUBLISHER]

In 2004, Graña Drummond and Iusem proposed an extension of the projected gradient method for constrained vector optimization problems. Using this method, an Armijo-like rule, implemented with a backtracking procedure, was used in order to determine the step lengths. The authors just showed stationarity of all cluster points and, for another version of the algorithm (with exogenous step lengths), under some additional assumptions, they proved convergence to weakly efficient solutions. In this work, first we correct a slight mistake in the proof of a certain continuity result in that 2004 article, and then we extend its convergence analysis. Indeed, under some reasonable hypotheses, for convex objective functions with respect to the ordering cone, we establish full convergence to optimal points of any sequence produced by the projected gradient method with an Armijo-like rule, no matter how poor the initial guesses may be. [ABSTRACT FROM PUBLISHER]

Abstract: It is well-known that the PRP conjugate gradient method with exact line search is globally and linearly convergent. If a restart strategy is used, the convergence rate of the method can be an -step superlinear/quadratic convergence. Recently, Zhang et al. [L. Zhang, W. Zhou, D.H. Li, A descent modified Polak–Ribière–Polyak conjugate gradient method and its global convergence, IMA J. Numer. Anal. 26 (2006) 629–640] developed a modified PRP (MPRP) method that is globally convergent if an inexact line search is used. In this paper, we investigate the convergence rate of the MPRP method with inexact line search. We first show that the MPRP method with Armijo line search or Wolfe line search is linearly convergent. We then show that the MPRP method with a restart strategy still retains -step superlinear/quadratic convergence if the initial steplength is appropriately chosen. We also do some numerical experiments. The results show that the restart MPRP method does converge quadratically. Moreover, it is more efficient than the non-restart method. [Copyright &y& Elsevier]

Abstract: In this paper, a new spectral PRP conjugate gradient algorithm has been developed for solving unconstrained optimization problems, where the search direction was a kind of combination of the gradient and the obtained direction, and the steplength was obtained by the Wolfe-type inexact line search. It was proved that the search direction at each iteration is a descent direction of objective function. Under mild conditions, we have established the global convergence theorem of the proposed method. Numerical results showed that the algorithm is promising, particularly, compared with the existing several main methods. [ABSTRACT FROM AUTHOR]

We introduced an algorithm for unconstrained optimization based on the transformation of the Newton method with the line search into a gradient descent method. Main idea used in the algorithm construction is approximation of the Hessian by an appropriate diagonal matrix. The steplength calculation algorithm is based on the Taylor’s development in two successive iterative points and the backtracking line search procedure. The linear convergence of the algorithm is proved for uniformly convex functions and strictly convex quadratic functions satisfying specified conditions. [ABSTRACT FROM AUTHOR]

Abstract: This paper presents a modified quasi-Newton method for structured unconstrained optimization. The usual SQN equation employs only the gradients, but ignores the available function value information. Several researchers paid attention to other secant conditions to get a better approximation of the Hessian matrix of the objective function. Recently Yabe et al. (2007) proposed the modified secant condition which uses both gradient and function value information in order to get a higher-order accuracy in approximating the second curvature of the objective function. In this paper, we derive a new progressive modified SQN equation, with a vector parameter which use both available gradient and function value information, that maintains most properties of the usual and modified structured quasi-Newton methods. Furthermore, local and superlinear convergence of the algorithm is obtained under some reasonable conditions. [Copyright &y& Elsevier]

Abstract  A method for linearly constrained optimization which modifies and generalizes recent box-constraint optimization algorithms is introduced. The new algorithm is based on a relaxed form of Spectral Projected Gradient iterations. Intercalated with these projected steps, internal iterations restricted to faces of the polytope are performed, which enhance the efficiency of the algorithm. Convergence proofs are given and numerical experiments are included and commented. Software supporting this paper is available through the Tango Project web page: http://www.ime.usp.br/∼egbirgin/tango/. [ABSTRACT FROM AUTHOR]

Abstract: A modified conjugate gradient method is presented for solving unconstrained optimization problems, which possesses the following properties: (i) The sufficient descent property is satisfied without any line search; (ii) The search direction will be in a trust region automatically; (iii) The Zoutendijk condition holds for the Wolfe–Powell line search technique; (iv) This method inherits an important property of the well-known Polak–Ribière–Polyak (PRP) method: the tendency to turn towards the steepest descent direction if a small step is generated away from the solution, preventing a sequence of tiny steps from happening. The global convergence and the linearly convergent rate of the given method are established. Numerical results show that this method is interesting. [Copyright &y& Elsevier]

In this paper, we study methods for generating approximate primal solutions as a byproduct of subgradient methods applied to the Lagrangian dual of a primal convex (possibly nondifferentiable) constrained optimization problem. Our work is motivated by constrained primal problems with a favorable dual problem structure that leads to efficient implementation of dual subgradient methods, such as the recent resource allocation problems in large-scale networks. For such problems, we propose and analyze dual subgradient methods that use averaging schemes to generate approximate primal optimal solutions. These algorithms use a constant stepsize in view of its simplicity and practical significance. We provide estimates on the primal infeasibility and primal suboptimality of the generated approximate primal solutions. These estimates are given per iteration, thus providing a basis for analyzing the trade-offs between the desired level of error and the selection of the stepsize value. Our analysis relies on the Slater condition and the inherited boundedness properties of the dual problem under this condition. It also relies on the boundedness of subgradients, which is ensured by assuming the compactness of the constraint set. [ABSTRACT FROM AUTHOR]

This paper proposes a line search technique to satisfy a relaxed form of the strong Wolfe conditions in order to guarantee the descent condition at each iteration of the Polak-Ribière-Polyak conjugate gradient algorithm. It is proved that this line search algorithm preserves the usual convergence properties of any descent algorithm. In particular, it is shown that the Zoutendijk condition holds under mild assumptions. It is also proved that the resulting conjugate gradient algorithm is convergent under a strong convexity assumption. For the nonconvex case, a globally convergent modification is proposed. Numerical tests are presented. [ABSTRACT FROM AUTHOR]

A popular approach to solving the Karush--Kuhn--Tucker (KKT) system, mainly arising from the variational inequality problem, is to reformulate it as a constrained minimization problem with simple bounds. In this paper, we propose a trust region method for solving the reformulation problem with the trust region subproblems being solved by the truncated conjugate gradient (CG) method, which is cost effective. Other advantages of the proposed method over existing ones include the fact that a good approximated solution to the trust region subproblem can be found by the truncated CG method and is judged in a simple way; also, the working matrix in each iteration is H , instead of the condensed HT H , where H is a matrix element of the generalized Jacobian of the function used in the reformulation. As a matter of fact, the matrix used is of reduced dimension. We pay extra attention to ensure the success of the truncated CG method as well as the feasibility of the iterates with respect to the simple constraints. Another feature of the proposed method is that we allow the merit function value to be increased at some iterations to speed up the convergence. Global and superlinear/quadratic convergence is shown under standard assumptions. Numerical results are reported on a subset of problems from the MCPLIB... [ABSTRACT FROM AUTHOR]