Search

The current study provides two modified adaptive cubic regularized methods for the unconstrained optimization problems. The basis of these methods are based on a combination of the adaptive cubic regularized method (ARC) with the nonmonotone Armijo-type line search. The step lengths are calculated in different paths for the modified algorithms. Moreover, we analyse the global convergence properties for the proposed methods. The obtained results illustrate the capability of the modified algorithms compared to the other provided algorithms. [ABSTRACT FROM AUTHOR]

Founded upon a sparse estimation of the Hessian obtained by a recent diagonal quasi-Newton update, a conjugacy condition is given, and then, a class of conjugate gradient methods is developed, being modifications of the Hestenes–Stiefel method. According to the given sparse approximation, the curvature condition is guaranteed regardless of the line search technique. Convergence analysis is conducted without convexity assumption, based on a nonmonotone Armijo line search in which a forgetting factor is embedded to enhance probability of applying more recent available information. Practical advantages of the method are computationally depicted on a set of CUTEr test functions and also, on the well-known signal processing problems such as sparse recovery and nonnegative matrix factorization. [ABSTRACT FROM AUTHOR]

In this paper, we consider the composite optimization problems over the Stiefel manifold. A successful method to solve this class of problems is the proximal gradient method proposed by Chen et al. (SIAM J Optim 30:210–239, 2020. https://doi.org/10.1137/18M122457X). Motivated by the proximal Newton-type techniques in the Euclidean space, we present a Riemannian proximal quasi-Newton method, named ManPQN, to solve the composite optimization problems. The global convergence of the ManPQN method is proved and iteration complexity for obtaining an ϵ -stationary point is analyzed. Under some mild conditions, we also establish the local linear convergence result of the ManPQN method. Numerical results are encouraging, which shows that the proximal quasi-Newton technique can be used to accelerate the proximal gradient method. [ABSTRACT FROM AUTHOR]

We consider a projected gradient method equipped with the nonmonotone line search procedure for convex constrained multiobjective optimization problems. Under mild assumptions, we show the convergence of the full sequence generated by the algorithm to a weak Pareto optimal point. Furthermore, under some appropriate Lipschitz continuity assumption of the gradients of objective functions, a linear convergence result for this method is also established. [ABSTRACT FROM AUTHOR]

In this paper, two new subspace minimization conjugate gradient methods based on p-regularization models are proposed, where a special scaled norm in p-regularization model is analyzed. Different choices of special scaled norm lead to different solutions to the p-regularized subproblem. Based on the analyses of the solutions in a two-dimensional subspace, we derive new directions satisfying the sufficient descent condition. With a modified nonmonotone line search, we establish the global convergence of the proposed methods under mild assumptions. R-linear convergence of the proposed methods is also analyzed. Numerical results show that, for the CUTEr library, the proposed methods are superior to four conjugate gradient methods, which were proposed by Hager and Zhang (SIAM J. Optim. 16(1):170–192, 2005), Dai and Kou (SIAM J. Optim. 23(1):296–320, 2013), Liu and Liu (J. Optim. Theory. Appl. 180(3):879–906, 2019) and Li et al. (Comput. Appl. Math. 38(1):2019), respectively. [ABSTRACT FROM AUTHOR]

This article is concerned with the problem of minimizing a smooth function over the Stiefel manifold. In order to address this problem, we introduce two adaptive scaled gradient projection methods that incorporate scaling matrices that depend on the step-size and a parameter that controls the search direction. These iterative algorithms use a projection operator based on the QR factorization to preserve the feasibility in each iteration. However, for some particular cases, the proposals do not require the use of any projection operator. In addition, we consider a Barzilai and Borwein-like step-size combined with the Zhang–Hager nonmonotone line-search technique in order to accelerate the convergence of the proposed procedures. We proved the global convergence for these schemes, and we evaluate their effectiveness and efficiency through an extensive computational study, comparing our approaches with other state-of-the-art gradient-type algorithms. [ABSTRACT FROM AUTHOR]

In this paper, we present two nonmonotone versions of adaptive cubic regularized (ARC) method for unconstrained optimization problems. The proposed methods are a combination of the ARC algorithm with the nonmonotone line search methods introduced by Zhang and Hager [A nonmonotone line search technique and its application to unconstrained optimization, SIAM J. Optim. 14 (2004), pp. 1043–1056] and Ahookhosh et al. [A nonmonotone trust-region line search method for large-scale unconstrained optimization, Appl. Math. Model. 36 (2012), pp. 478–487]. The global convergence analysis for these iterative algorithms is established under suitable conditions. Several numerical examples are given to illustrate the efficiency and robustness of the newly suggested methods. The obtained results show the satisfactory performance of the proposed algorithms when compared to the basic ARC algorithm. [ABSTRACT FROM AUTHOR]

In this paper, we develop a new algorithmic framework to solve black-box problems with integer variables. The strategy included in the framework makes use of specific search directions (so called primitive directions) and a suitably developed nonmonotone line search, thus guaranteeing a high level of freedom when exploring the integer lattice. First, we describe and analyze a version of the algorithm that tackles problems with only bound constraints on the variables. Then, we combine it with a penalty approach in order to solve problems with simulation constraints. In both cases we prove finite convergence to a suitably defined local minimum of the problem. We report extensive numerical experiments based on a test bed of both bound-constrained and generally-constrained problems. We show the effectiveness of the method when compared to other state-of-the-art solvers for black-box integer optimization. [ABSTRACT FROM AUTHOR]

In this paper we study the nonlinear weighted complementarity problem (denoted by NWCP). We first introduce a smoothing function which can be used to reformulate the NWCP as a system of smooth nonlinear equations. Then we propose a new smoothing-type algorithm to solve the NWCP which adopts a nonmonotone line search scheme. In each iteration, our algorithm solves one linear system of equations and performs one line search. Under suitable assumptions, we prove that the proposed algorithm is globally and locally quadratically convergent. Some numerical results are reported. [ABSTRACT FROM AUTHOR]

In this paper, we present a QP-free algorithm without a penalty function or a filter for nonlinear semideflnite programming. At each iteration, two systems of linear equations with the same coefficient matrix are solved to determine search direction; the nonmonotone line search ensures that the objective function or constraint violation function is sufficiently reduced. There is no feasibility restoration phase in our algorithm, which is necessary for traditional filter methods. The proposed algorithm is globally convergent under some mild conditions. Preliminary numerical results indicate that the proposed algorithm is comparable. [ABSTRACT FROM AUTHOR]

A new subspace minimization conjugate gradient algorithm with a nonmonotone Wolfe line search is proposed and analyzed. In the scheme, we propose two choices of the search direction by minimizing a quadratic approximation of the objective function in special subspaces, and state criterions on how to choose the direction. Under given conditions, we obtain the significant conclusion that each choice of the direction satisfies the sufficient descent property. Based on the idea on how the function is close to a quadratic function, a new strategy for choosing the initial stepsize is presented for the line search. With the used nonmonotone Wolfe line search, we prove the global convergence of the proposed method for general nonlinear functions under mild assumptions. Numerical comparisons are given with well-known CGOPT and CG_DESCENT and show that the proposed algorithm is very promising. [ABSTRACT FROM AUTHOR]

In this paper, we present a new spectral residual method for solving large-scale positive definite generalized eigenvalue problems. The proposed algorithm is equipped with a dwindling multidimensional nonmonotone filter, in which the envelope dwindles as the step length decreases. We have also employed a relaxed nonmonotone line search technique in the structure of the algorithm which allows it to enjoy the nonmonotonicity from scratch. Under some mild and standard assumptions, the global convergence property of the proposed algorithm is established. An implementation of the new algorithm on some test problems shows the efficiency and effectiveness of the proposed algorithm in practice. [ABSTRACT FROM AUTHOR]

In this paper, an extended nonmonotone line search is proposed to improve the efficiency of the existing line searches. This line search is first proved to be an extension of the classical line search rules. On the one hand, under mild assumptions, global convergence and R-linear convergence are established for the new line search rule. On the other hand, by numerical experiments, it is shown that the line search can integrate the advantages of the existing methods in searching for a suitable step-size. Combined with the spectral step-size, a class of spectral gradient algorithms are developed and employed to solve a large number of benchmark test problems from CUTEst. Numerical results show that the new line search is promising in solving large-scale optimization problems, and outperforms the other similar ones as it is combined with a spectral gradient method. [ABSTRACT FROM AUTHOR]

The nonmonotone globalization technique is useful in difficult nonlinear problems, because of the fact that it may help escaping from steep sided valleys and may improve both the possibility of finding the global optimum and the rate of convergence. This paper discusses the nonmonotonicity degree of nonmonotone line searches for the unconstrained optimization. Specifically, we analyze some popular nonmonotone line search methods and explore, from a computational point of view, the relations between the efficiency of a nonmonotone line search and its nonmonotonicity degree. We attempt to answer this question how to control the degree of the nonmonotonicity of line search rules in order to reach a more efficient algorithm. Hence in an attempt to control the nonmonotonicity degree, two adaptive nonmonotone rules based on the morphology of the objective function are proposed. The global convergence and the convergence rate of the proposed methods are analysed under mild assumptions. Numerical experiments are made on a set of unconstrained optimization test problems of the CUTEr (Gould et al. in ACM Trans Math Softw 29:373-394, 2003) collection. The performance data are first analysed through the performance profile of Dolan and Moré (Math Program 91:201-213, 2002). In the second kind of analyse, the performance data are analysed in terms of increasing dimension of the test problems. [ABSTRACT FROM AUTHOR]

This paper presents a nonmonotone inexact Newton-type method for the extended linear complementarity problem (ELCP). We first reformulate the optimization system of the ELCP problem into a system of smoothed equations. Then we solve this system by a nonmonotone inexact Newton-type algorithm. The global convergence is obtained and numerical tests for some classes of ELCP include linear complementarity, horizontal linear complementarity, and generalized linear complementarity problems are also given to show the efficiency of the proposed algorithm. [ABSTRACT FROM PUBLISHER]

In this paper, a new spectral residual method is proposed to solve systems of large-scale nonlinear equations, where the steplength is obtained by minimizing the residue of an approximate secant equation. Especially, the new steplength can be directly applied into solving strictly convex quadratic function. Combined with a new nonmonotone line search strategy, a new derivative-free algorithm, called a nonmonotone spectral residual algorithm (NSRA), is developed. Under mild assumptions, global convergence is established for locally Lipschitz continuous nonlinear systems. Compared with the state-of-the-art algorithms available in the literatures, the new algorithm is more efficient in solving large-scale benchmark test problems. [ABSTRACT FROM AUTHOR]

The circular cone programming (CCP) problem is to minimize or maximize a linear function over the intersection of an affine space with the Cartesian product of circular cones. In this paper, we study nondegeneracy and strict complementarity for the CCP, and present a nonmonotone smoothing Newton method for solving the CCP. We reformulate the CCP as a second-order cone programming (SOCP) problem using the algebraic relation between the circular cone and the second-order cone. Then based on a one parametric class of smoothing functions for the SOCP, a smoothing Newton method is developed for the CCP by adopting a new nonmonotone line search scheme. Without restrictions regarding its starting point, our algorithm solves one linear system of equations approximately and performs one line search at each iteration. Under mild assumptions, our algorithm is shown to possess global and local quadratic convergence properties. Some preliminary numerical results illustrate that our nonmonotone smoothing Newton method is promising for solving the CCP. [ABSTRACT FROM PUBLISHER]

Based on the Moreau-Yosida regularization and a modified nonmonotone line search technique, this paper presents an implementable ordinary differentiable equation-like method for solving a possibly nondifferentiable convex minimization problem by converting the original objective function to a once continuously differentiable function. The proposed method makes use of approximate function and gradient values of the Moreau-Yosida regularization instead of the corresponding exact values. Under some reasonable conditions, the proposed method is shown to be globally and superlinearly convergent. Some preliminary numerical results are also reported to show the efficiency of the proposed method. [ABSTRACT FROM AUTHOR]

In this paper, a new nonmonotone line search rule is proposed,which is verified to be an improved version of the nonmonotone line search technique proposed by Zhang and Hager. Unlike the Zhang and Hager's method, our nonmonotone line search is proved to own a nice property similar to the standard Armijo line search. In virtue of such a property, global convergence is established for the developed algorithm, where the search direction is supposed to satisfy some mild conditions and the stepsize is chosen by the new line search rule. R-linear convergence of the developed algorithm is proved for strongly convex objective functions. The developed algorithm is used to solve the test problems available in the CUTEr, the numerical results demonstrate that the new line search strategy outperforms the other similar ones. [ABSTRACT FROM AUTHOR]

Abstract: An implementable proximal point algorithm is established for the smooth nonconvex unconstrained minimization problem. At each iteration, the algorithm minimizes approximately a general quadratic by a truncated strategy with step length control. The main contributions are: (i) a framework for updating the proximal parameter; (ii) inexact criteria for approximately solving the subproblems; (iii) a nonmonotone criterion for accepting the iterate. The global convergence analysis is presented, together with numerical results that validate and put into perspective the proposed approach. [Copyright &y& Elsevier]

This paper presents a nonmonotone supermemory gradient algorithm for unconstrained optimization problems. At each iteration, this proposed method sufficiently uses the previous multi-step iterative information and avoids the storage and computation of matrices associated with the Hessian of objective functions, thus it is suitable to solve large-scale optimization problems and can converge stably. Under some assumptions, the convergence properties of the proposed algorithm are analyzed. Numerical results are also reported to show the efficiency of this proposed method. [ABSTRACT FROM AUTHOR]

This paper presents a new hybrid algorithm for unconstrained optimization problems, which combines the idea of the IMPBOT algorithm with the nonmonotone line search technique. A feature of the proposed method is that at each iteration, a system of linear equations is solved only once to obtain a trial step, via a modified limited-memory BFGS two loop recursion that requires only matrix–vector products, thus reducing the computations and storage. Furthermore, when the trial step is not accepted, the proposed method performs a line search along it using a modified nonmonotone scheme, thus a larger stepsize can be yielded in each line search procedure. Under some reasonable assumptions, the convergence properties of the proposed algorithm are analysed. Numerical results are also reported to show the efficiency of this proposed method. [ABSTRACT FROM PUBLISHER]

We consider equilibrium problems with differentiable bifunctions. We adopt the well-known approach based on the reformulation of the equilibrium problem as a global optimization problem through an appropriate gap function. We propose a solution method based on the inexact (and hence, less expensive) evaluation of the gap function and on the employment of a nonmonotone line search. We prove global convergence properties of the proposed inexact method under standard assumptions. Some preliminary numerical results show the potential computational advantages of the inexact method compared with a standard exact descent method. [ABSTRACT FROM AUTHOR]

Abstract: The conjugate gradient (CG) method is one of the most popular methods for solving smooth unconstrained optimization problems due to its simplicity and low memory requirement. However, the usage of CG methods is mainly restricted to solving smooth optimization problems so far. The purpose of this paper is to present efficient conjugate gradient-type methods to solve nonsmooth optimization problems. By using the Moreau–Yosida regulation (smoothing) approach and a nonmonotone line search technique, we propose a modified Polak–Ribière–Polyak (PRP) CG algorithm for solving a nonsmooth unconstrained convex minimization problem. Our algorithm possesses the following three desired properties. (i) The search direction satisfies the sufficient descent property and belongs to a trust region automatically; (ii) the search direction makes use of not only the gradient information but also the function value information; and (iii) the algorithm inherits an important property of the well-known PRP method: the tendency to turn towards the steepest descent direction if a small step is generated away from the solution, preventing a sequence of tiny steps from happening. Under standard conditions, we show that the algorithm converges globally to an optimal solution. Numerical experiment shows that our algorithm is effective and suitable for solving large-scale nonsmooth unconstrained convex optimization problems. [Copyright &y& Elsevier]

This paper proposes an ODE-based nonmonotone method for unconstrained optimization problems, which combines the idea of IMPBOT with the nonmonotone technique. The main characteristic of this method is that at each iteration, a system of linear equations is solved only once to obtain a trial step, via a modified L-BFGS two loop recursion that requires only vector inner products, thus reducing the matrix computation and storage. Then a modified nonmonotone line search is performed to generate next iterative point instead of resolving the linear system. Under some reasonable assumptions, the method is proven to be globally and superlinearly convergent. Numerical results show the efficiency of this proposed method in practical computation. [ABSTRACT FROM AUTHOR]

Nonmonotone line search approach is a new technique for solving optimization problems. It relaxes the line search range and finds a larger step-size at each iteration, so as to possibly avoid local minimizer and run away from narrow curved valley. It is helpful to find the global minimizer of optimization problems. In this paper we develop a new modification of matrix-free nonmonotone Armijo line search and analyze the global convergence and convergence rate of the resulting method. We also address several approaches to estimate the Lipschitz constant of the gradient of objective functions that would be used in line search algorithms. Numerical results show that this new modification of Armijo line search is efficient for solving large scale unconstrained optimization problems. [ABSTRACT FROM AUTHOR]

Abstract: In this paper, we consider a multivariate spectral projected gradient (MSPG) method for bound constrained optimization. Combined with a quasi-Newton property, the multivariate spectral projected gradient method allows an individual adaptive step size along each coordinate direction. On the basis of nonmonotone line search, global convergence is established. A numerical comparison with the traditional SPG method shows that the method is promising. [ABSTRACT FROM AUTHOR]

Abstract: In this paper, we propose a new nonmonotone Armijo type line search and prove that the MBFGS method proposed by Li and Fukushima with this new line search converges globally for nonconvex minimization. Some numerical experiments show that this nonmonotone MBFGS method is efficient for the given test problems. [Copyright &y& Elsevier]

The self-scaling quasi-Newton method solves an unconstrained optimization problem by scaling the Hessian approximation matrix before it is updated at each iteration to avoid the possible large eigenvalues in the Hessian approximation matrices of the objective function. It has been proved in the literature that this method has the global and superlinear convergence when the objective function is convex (or even uniformly convex). We propose to solve unconstrained nonconvex optimization problems by a self-scaling BFGS algorithm with nonmonotone linear search. Nonmonotone line search has been recognized in numerical practices as a competitive approach for solving large-scale nonlinear problems. We consider two different nonmonotone line search forms and study the global convergence of these nonmonotone self-scale BFGS algorithms. We prove that, under some weaker condition than that in the literature, both forms of the self-scaling BFGS algorithm are globally convergent for unconstrained nonconvex optimization problems. [ABSTRACT FROM AUTHOR]

Abstract: In this paper, we develop a new nonmonotone line search for general line search method and establish some global convergence theorems. The new nonmonotone line search is a novel form of the nonmonotone Armijo line search and allows one to choose a larger step size at each iteration, which is available in constructing new line search methods and possibly reduces the function evaluations at each iteration. Moreover, we analyze the convergence rate of some special line search methods with the new line search. Preliminary numerical results show that some line search methods with the new nonmonotone line search are available and efficient in practical computation. [Copyright &y& Elsevier]

This paper discusses the global convergence of a class of nonmonotone conjugate gradient methods (NM methods) for nonconvex object functions. This class of methods includes the nonmonotone counterpart of modified Polak-Ribière method and modified Hestenes-Stiefel method as special cases. [ABSTRACT FROM AUTHOR]

In this paper, we present a new conjugate gradient method, in which the search direction is computed by minimizing a selected approximate model in a two-dimensional subspace. That is, if the objective function is not close to a quadratic, the search direction is generated by a conic model. Otherwise, a quadratic model is considered. The direction of the proposed method is proved to possess the sufficient descent property. With the modified nonmonotone line search, we establish a global convergence of the proposed method under appropriate assumptions. R-linear convergence of the proposed method is also analyzed. Numerical results using two different test function collections show that the proposed algorithm is efficient. [ABSTRACT FROM AUTHOR]