Search

Abstract The role of the steplength selection strategies in gradient methods has been widely investigated in the last decades. Starting from the work of Barzilai and Borwein (1988), many efficient steplength rules have been designed, that contributed to make the gradient approaches an effective tool for the large-scale optimization problems arising in important real-world applications. Most of these steplength rules have been thought in unconstrained optimization, with the aim of exploiting some second-order information for achieving a fast annihilation of the gradient of the objective function. However, these rules are successfully used also within gradient projection methods for constrained optimization, though, to our knowledge, a detailed analysis of the effects of the constraints on the steplength selections is still not available. In this work we investigate how the presence of the box constraints affects the spectral properties of the Barzilai–Borwein rules in quadratic programming problems. The proposed analysis suggests the introduction of new steplength selection strategies specifically designed for taking account of the active constraints at each iteration. The results of a set of numerical experiments show the effectiveness of the new rules with respect to other state of the art steplength selections and their potential usefulness also in case of box-constrained non-quadratic optimization problems. [ABSTRACT FROM AUTHOR]

This paper presents a project for three-term conjugate gradient algorithms development. The search direction of the algorithms from this class has three terms and is computed as modifications of the classical conjugate gradient algorithms to satisfy both the descent and the conjugacy conditions. An example of three-term conjugate gradient algorithm from this class, as modifications of the classical and well known Hestenes and Stiefel or of the CG_DESCENT by Hager and Zhang conjugate gradient algorithms, satisfying both the descent and the conjugacy conditions is presented. These properties are independent of the line search. Also, the algorithm could be considered as a modification of the memoryless BFGS quasi-Newton method. The new approximation of the minimum is obtained by the general Wolfe line search and using by now a standard acceleration technique developed by Andrei. The proposed three-term conjugate gradient algorithm substantially outperforms the well known Hestenes and Stiefel conjugate gradient algorithm, as well as the more elaborated CG_DESCENT algorithm. A number of 5 applications from MINPACK-2 test problems collection, with 106 variables, prove that the suggested three-term conjugate gradient algorithm is top performer versus CG_DESCENT. [ABSTRACT FROM AUTHOR]

Abstract: In this paper, we consider the Polak–Ribière (or Polak–Ribière plus) conjugate gradient method for solving optimality condition of an unconstrained minimization problem. We give two new steplength rules only using gradient, and under gradient-Lipschitz assumption prove this method’s global convergence correspondingly. Then, we develop a practical Polak–Ribière plus method whose steplength is located by one inequality only using gradient, and report promising numerical results on high accuracy solution for some standard test problems when compared to the state-of-art methods in this research direction. Importantly, our work provides a new idea of devising a practical version of the celebrated Polak–Ribière (or Polak–Ribière plus) method. [Copyright &y& Elsevier]

Abstract: Simple modifications of the limited-memory BFGS method (L-BFGS) for large scale unconstrained optimization are considered, which consist in corrections (derived from the idea of conjugate directions) of the used difference vectors, utilizing information from the preceding iteration. For quadratic objective functions, the improvement of convergence is the best one in some sense and all stored difference vectors are conjugate for unit stepsizes. Global convergence of the algorithm is established for convex sufficiently smooth functions. Numerical experiments indicate that the new method often improves the L-BFGS method significantly. [Copyright &y& Elsevier]

Abstract: Nonlinear conjugate gradient (CG) methods have played an important role in solving large-scale unconstrained optimization. Their wide application in many fields is due to their low memory requirements and global convergence properties. Numerous studies and modifications have been conducted recently to improve this method. In this paper, a new class of conjugate gradient coefficients (βk ) that possess global convergence properties is presented. The global convergence result is established using exact line searches. Numerical result shows that the proposed formula is superior and more efficient when compared to other CG coefficients. [Copyright &y& Elsevier]

Abstract: We introduce a new algorithm for the unconstrained optimization, based on introducing the random parameter θ ∈(0,1) in the standard iterative scheme for the descent gradient method. Thus, some recent results are improved. [Copyright &y& Elsevier]