Search

The paper is devoted to the theoretical and numerical analysis of the two-step method, constructed as a modification of Polyak's heavy ball method with the inclusion of an additional momentum parameter. For the quadratic case, the convergence conditions are obtained with the use of the first Lyapunov method. For the non-quadratic case, sufficiently smooth strongly convex functions are obtained, and these conditions guarantee local convergence.An approach to finding optimal parameter values based on the solution of a constrained optimization problem is proposed. The effect of an additional parameter on the convergence rate is analyzed. With the use of an ordinary differential equation, equivalent to the method, the damping effect of this parameter on the oscillations, which is typical for the non-monotonic convergence of the heavy ball method, is demonstrated. In different numerical examples for non-quadratic convex and non-convex test functions and machine learning problems (regularized smoothed elastic net regression, logistic regression, and recurrent neural network training), the positive influence of an additional parameter value on the convergence process is demonstrated. [ABSTRACT FROM AUTHOR]

The motivation for this paper stems from the desire to develop an adaptive sampling method for solving constrained optimization problems, in which the objective function is stochastic and the constraints are deterministic. The method proposed in this paper is a proximal gradient method that can also be applied to the composite optimization problem min |$f(x) + h(x)$|⁠ , where |$f$| is stochastic and |$h$| is convex (but not necessarily differentiable). Adaptive sampling methods employ a mechanism for gradually improving the quality of the gradient approximation so as to keep computational cost to a minimum. The mechanism commonly employed in unconstrained optimization is no longer reliable in the constrained or composite optimization settings, because it is based on pointwise decisions that cannot correctly predict the quality of the proximal gradient step. The method proposed in this paper measures the result of a complete step to determine if the gradient approximation is accurate enough; otherwise, a more accurate gradient is generated and a new step is computed. Convergence results are established both for strongly convex and general convex |$f$|⁠. Numerical experiments are presented to illustrate the practical behavior of the method. [ABSTRACT FROM AUTHOR]

In this paper, we focus on solving a distributed convex aggregative optimization problem in a network, where each agent has its own cost function which depends not only on its own decision variables but also on the aggregated function of all agents' decision variables. The decision variable is constrained within a feasible set. In order to minimize the sum of the cost functions when each agent only knows its local cost function, we propose a distributed Frank–Wolfe algorithm based on gradient tracking for the aggregative optimization problem where each node maintains two estimates, namely an estimate of the sum of agents' decision variable and an estimate of the gradient of global function. The algorithm is projection‐free, but only involves solving a linear optimization to get a search direction at each step. We show the convergence of the proposed algorithm for convex and smooth objective functions over a time‐varying network. Finally, we demonstrate the convergence and computational efficiency of the proposed algorithm via numerical simulations. [ABSTRACT FROM AUTHOR]

The stochastic inverse eigenvalue problem aims to reconstruct a stochastic matrix from its spectrum. Recently, Zhao et al. [A geometric nonlinear conjugate gradient method for stochastic inverse eigenvalue problems. SIAM J Numer Anal. 2016;54(4):2015–2035] proposed a constrained optimization model on the manifold of so-called isospectral matrices and adapted a modified. Polak-Ribière-Polyak conjugate gradient method to the geometry of this manifold. However, not every stochastic matrix is an isospectral one and the model in Zhao et al. is based on the assumption that for each stochastic matrix there exists a (possibly different) isospectral, stochastic matrix with the same spectrum. We are not aware of such a result in the literature, but prove the claim at least for 3 × 3 matrices. In this paper, we suggest to extend the above model by considering matrices which differ from isospectral ones only by multiplication with a block diagonal matrix with 2 × 2 blocks from the special linear group. First, we show that each stochastic matrix can be written in such a form. We prove that our model has a minimizer and show how the Polak–Ribiére–Polyak conjugate gradient method works on the corresponding more general manifold. We demonstrate by numerical examples that the new, more general method performs similarly as those in Zhao et al. [ABSTRACT FROM AUTHOR]

Given several nonnegative matrices with a single pattern of allocation among their zero/nonzero elements, the average matrix should have the same pattern as well. This is the first tenet of the pattern-multiplicative average (PMA) concept, while the second one suggests the multiplicative nature of averaging. The concept of PMA was motivated in a number of application fields, of which we consider the matrix population models and illustrate solving the PMA problem with several sets of model matrices calibrated in particular botanic case studies. The patterns of those matrices are typically nontrivial (they contain both zero and nonzero elements), the PMA problem thus having no exact solution for a fundamental reason (an overdetermined system of algebraic equations). Therefore, searching for the approximate solution reduces to a constrained minimization problem for the approximation error, the loss function in optimization terms. We consider two alternative types of the loss function and present a general algorithm of searching the optimal solution: basin-hopping global search, then local descents by the method of conjugate gradients or that of penalty functions. Theoretical disadvantages and practical limitations of both loss functions are discussed and illustrated with a number of practical examples. [ABSTRACT FROM AUTHOR]

In this paper, we propose a new estimation methodology based on a projected non-linear conjugate gradient (PNCG) algorithm with an efficient line search technique. We develop a general PNCG algorithm for a survival model incorporating a proportion cure under a competing risks setup, where the initial number of competing risks are exposed to elimination after an initial treatment (known as destruction). In the literature, expectation maximization (EM) algorithm has been widely used for such a model to estimate the model parameters. Through an extensive Monte Carlo simulation study, we compare the performance of our proposed PNCG with that of the EM algorithm and show the advantages of our proposed method. Through simulation, we also show the advantages of our proposed methodology over other optimization algorithms (including other conjugate gradient type methods) readily available as R software packages. To show these, we assume the initial number of competing risks to follow a negative binomial distribution although our general algorithm allows one to work with any competing risks distribution. Finally, we apply our proposed algorithm to analyze a well-known melanoma data. [ABSTRACT FROM AUTHOR]

We propose a novel preconditioned inexact primal-dual interior point method for constrained convex quadratic programming problems. The algorithm we describe invokes the preconditioned conjugate gradient method on a new reduced Schur complement Karush-Kuhn-Tucker (KKT) system in implicit form. In contrast to standard approaches, the Schur complement formulation we consider enables reuse of the factorization of the KKT matrix with rows and columns corresponding to inequality constraints excluded across all interior point iterations. Further, two new preconditioners are presented for the resulting reduced system that alleviate the ill-conditioning associated with slack variables in primal-dual interior point methods. Each of the preconditioners we propose also provably reduces the number of unique eigenvalues for the coefficient matrix and thus the conjugate gradient method iteration count. One preconditioner is efficient when the number of equality constraints is small, while the other is efficient when the number of remaining degrees of freedom is small. Numerical experiments with synthetic problems and problems from the Maros-Mészáros quadratic programming collection show that our preconditioned inexact interior point solvers are effective at improving conditioning and reducing cost. Across all test problems for which the direct method is not fastest, our preconditioned methods achieve a reduction in cost by a geometric mean of 1.432 relative to the best alternative preconditioned method for each problem. [ABSTRACT FROM AUTHOR]

This study analyzes the conditional gradient method for constrained multiobjective optimization problems, also known as the Frank–Wolfe method. We assume that the objectives are continuously differentiable, and the constraint set is convex and compact. We employ an average-type nonmonotone line search, which takes the average of the recent objective function values. The asymptotic convergence properties without convexity assumptions on the objective functions are established. We prove that every limit point of the sequence of iterates that is obtained by the proposed method is a Pareto critical point. An iteration-complexity bound is provided regardless of the convexity assumption on the objective functions. The effectiveness of the suggested approach is demonstrated by applying it to several benchmark test problems. In addition, the efficiency of the proposed algorithm in generating approximations of the entire Pareto front is compared to the existing Hager–Zhang conjugate gradient method, the steepest descent method, the monotone conditional gradient method, and a nonmonotone conditional gradient method. In finding empirical comparison, we utilize two commonly used performance matrices—inverted generational distance and hypervolume indicators. [ABSTRACT FROM AUTHOR]

One of the important techniques used for solving the constrained and unconstrained optimization problems is the conjugate gradient CG method, which is one of the efficient techniques used for solving the unconstrained optimization problems due to its advantages of ease of application and its lack of need for large storage capacities as well as its ability at the same time to solve problems that need a high storage capacity (large scale). In this work, this method was modified by combining βk parameters of the Polak-Ribiere (PR) method and βk of the Hestenes-Stiefel (HS) method to solve these kinds of problems. By comparing this method with three famous algorithms, using the Matlab program, the numerical expirement showed that our proposed method is the best. [ABSTRACT FROM AUTHOR]

In this paper, we propose two scaled Dai–Yuan (DY) directions for solving constrained monotone nonlinear systems. The proposed directions satisfy the sufficient descent condition independent of the line search strategy. We also reasonably proposed two different relations for computing the scaling parameter at every iteration. The first relation is proposed by approaching the quasi-Newton direction, and the second one is by taking the advantage of the popular Barzilai–Borwein strategy. Moreover, we propose a robust projection-based algorithm for solving constrained monotone nonlinear equations with applications in signal restoration problems and reconstructing the blurred images. The global convergence of this algorithm is also provided, using some mild assumptions. Finally, a comprehensive numerical comparison with the relevant algorithms shows that the proposed algorithm is efficient. [ABSTRACT FROM AUTHOR]

Over the last decades, many efforts have been made toward the understanding of the convergence rate of the gradient‐based method for both constrained and unconstrained optimization. The cases of the strongly convex and weakly convex payoff function have been extensively studied and are nowadays fully understood. Despite the impressive advances made in the convex optimization context, the nonlinear non‐convex optimization problems are still not fully exploited. In this paper, we are concerned with the nonlinear, non‐convex optimization problem under system dynamic constraints. We apply our analysis to parameter identification of systems governed by general nonlinear differential equations. The considered inverse problem is presented using optimal control tools. We tackle the optimization through the use of Fletcher‐Reeves nonlinear conjugate gradient method satisfying strong Wolfe conditions with inexact line search. We rigorously establish a convergence analysis of the method and report a new linear convergence rate which forms the main contribution of this work. The theoretical result reported in our analysis requires that the second derivative of the payoff functional be continuous and bounded. Numerical evidence on a selection of popular nonlinear models is presented as a direct application of parameter identification to support the theoretical findings. [ABSTRACT FROM AUTHOR]

In this paper, we derived a modified conjugate gradient (CG) parameter by adopting the Birgin and Mart i ´ nez strategy using the descent three-term CG direction and the Newton direction. The proposed CG parameter is applied and suggests a robust algorithm for solving constrained monotone equations with an application to image restoration problems. The global convergence of this algorithm is established using some proper assumptions. Lastly, the numerical comparison with some existing algorithms shows that the proposed algorithm is a robust approach for solving large-scale systems of monotone equations. Additionally, the proposed CG parameter can be used to solve the symmetric system of nonlinear equations as well as other relevant classes of nonlinear equations. [ABSTRACT FROM AUTHOR]

In this paper, a derivative-free affine scaling linear programming algorithm based on probabilistic models is considered for solving linear inequality constrainted optimization problems. The proposed algorithm is designed to build probabilistic linear polynomial interpolation models using only n + 1 interpolation points for the objective function and reduce the computation cost for building interpolation function. We build the affine scaling linear programming methods which use probabilistic or random models and affine matrix within a classical linear programming framework. The backtracking line search technique can guarantee monotone descent of the objective function, and by using this technique, the new iterative points are located within the feasible region. Under some reasonable conditions, the global and local fast convergence of the algorithm is shown, and the results of numerical experiments are reported to show the effectiveness of the proposed algorithm. [ABSTRACT FROM AUTHOR]

Vector extrapolations for fixed-point iterations are shown to converge faster when their step lengths are computed from two or three consecutive maps alternately. Based on this finding, cyclic extrapolation methods are proposed which require few objective function evaluations, no matrix inversion, and little extra memory. They are efficient in high-dimensional contexts and do not require problem-specific adaptation. A convergence analysis is done for symmetric positive definite linear systems and for contraction mappings. The proposed methods rivaled common quasi-Newton alternatives in eight mapping applications that included gradient descent for constrained and unconstrained optimization. • Vector extrapolation methods converge faster when alternating between 2 or 3 maps. • ACX require little computation and converge fast. • ACX accelerate gradient descent. • ACX are used for nonlinear optimization. • ACX accelerate the power method for dominant eigenvalues. [ABSTRACT FROM AUTHOR]

In recent times, various algorithms have been incorporated with the inertial extrapolation step to speed up the convergence of the sequence generated by these algorithms. As far as we know, very few results exist regarding algorithms of the inertial derivative-free projection method for solving convex constrained monotone nonlinear equations. In this article, the convergence analysis of a derivative-free iterative algorithm (Liu and Feng in Numer. Algorithms 82(1):245–262, 2019) with an inertial extrapolation step for solving large scale convex constrained monotone nonlinear equations is studied. The proposed method generates a sufficient descent direction at each iteration. Under some mild assumptions, the global convergence of the sequence generated by the proposed method is established. Furthermore, some experimental results are presented to support the theoretical analysis of the proposed method. [ABSTRACT FROM AUTHOR]

Process monitoring based on neural networks is getting more and more attention. Compared with classical neural networks, high-order neural networks have natural advantages in dealing with heteroscedastic data. However, high-order neural networks might bring the risk of overfitting, which learning both the key information from original data and noises or anomalies. Orthogonal constraints can greatly reduce correlations between extracted features, thereby reducing the overfitting risk. This paper proposes a novel fault detection method called second-order component analysis (SCA). SCA rules out the heteroscedasticity of process data by optimizing a second-order autoencoder with orthogonal constraints. In order to deal with this constrained optimization problem, a geometric conjugate gradient algorithm is adopted in this paper, which performs geometric optimization on the combination of Stiefel manifold and Euclidean manifold. Extensive experiments on the Tennessee-Eastman benchmark process show that SCA outperforms the compared state-of-the-art methods with missed detection rate (MDR) and false alarm rate (FAR). • Second-order component analysis (SCA) is proposed for fault detection. • SCA uses the structure of autoencoder and adds second-order terms to improve nonlinear mapping capabilities. • SCA adopts orthogonal constraints to reduce the overfitting problem. • SCA adopts a geometric conjugate gradient (GCG) algorithm to deal with constrained optimization problem. [ABSTRACT FROM AUTHOR]

The purposes of this paper are to establish and study the convergence of a new gradient scheme with penalization terms called rapid gradient penalty algorithm (RGPA) for minimizing a convex differentiable function over the set of minimizers of a convex differentiable constrained function. Under the observation of some appropriate choices for the available properties of the considered functions and scalars, we can generate a suitable algorithm that weakly converges to a minimal solution of the considered constraint minimization problem. Further, we also provide a numerical example to compare the rapid gradient penalty algorithm (RGPA) and the algorithm introduced by Peypouquet [20]. [ABSTRACT FROM AUTHOR]

A novel gradient stepsize is derived at the motivation of equipping the Barzilai--Borwein (BB) method with the two dimensional quadratic termination property. A remarkable feature of the novel stepsize is that its computation only depends on the BB stepsizes in previous iterations and does not require any exact line search or the Hessian, and hence it can easily be extended for nonlinear optimization. By adaptively taking long BB steps and some short steps associated with the new stepsize, we develop an efficient gradient method for quadratic optimization and general unconstrained optimization and extend it to solve extreme eigenvalue problems. The proposed method is further extended for box-constrained optimization and singly linearly box-constrained optimization by incorporating gradient projection techniques. Numerical experiments demonstrate that the proposed method outperforms the most successful gradient methods in the literature. [ABSTRACT FROM AUTHOR]

A new estimation method is proposed founded upon a nonlinear conjugate gradient‐type algorithm having an efficient line search technique for cure rate models with competing risks, which are subject to elimination. An extensive simulation study is carried out to compare the performance of the proposed algorithm with some existing algorithms, including other conjugate gradient‐type algorithms and the expectation maximization algorithm. For this purpose, it is assumed that the initial competing risks follow an exponentially weighted Poisson distribution. In particular, it is shown that that the proposed algorithm produces estimates that are more accurate and efficient (i.e., the bias and root mean square errors are smaller), specifically with respect to the parameters related to the cure rate. Although for the purpose of simulation study an exponentially weighted Poisson competing risks distribution is assumed, the proposed algorithm incorporates a generic framework that can accommodate any competing risks distribution. Finally, a real data application is provided. [ABSTRACT FROM AUTHOR]

Shape optimization based on shape calculus has received a lot of attention in recent years, particularly regarding the development, analysis, and modification of efficient optimization algorithms. In this paper we propose and investigate nonlinear conjugate gradient methods based on Steklov--Poincar\'e-type metrics for the solution of shape optimization problems constrained by partial differential equations. We embed these methods into a general algorithmic framework for gradientbased shape optimization methods and discuss the numerical discretization of the algorithms. We numerically compare the proposed nonlinear conjugate gradient methods to the already established gradient descent and limited memory BFGS methods for shape optimization on several benchmark problems. The results show that the proposed nonlinear conjugate gradient methods perform well in practice and that they are an efficient and attractive addition to already established gradient-based shape optimization algorithms. [ABSTRACT FROM AUTHOR]

It is widely known that strong noise can decrease the quality of seismic data. However, the anelastic attenuation could be more important to account for the weak amplitude and low quality of seismic data. Here, we develop an inversion framework to simultaneously compensate for the attenuation of seismic data and remove noise, thereby enhancing the quality of seismic data. Instead of directly applying a compensation operator to the input seismic data, we formulate an inverse problem that connects the sparse reflectivity model and the raw seismic data via the convolution and attenuation functions. The random noise is assumed to be the unpredicted part of the forward modeling process. We use the L2-norm regularization for the data misfit and impose a sparsity constraint onto the reflectivity series, e.g., using the L1-norm constraint. We use an iterative preconditioned conjugate gradient method to solve the L1-norm constrained least-squares optimization problem and obtain the reflectivity series. The denoised and compensated data are obtained by applying the convolution operator to the reflectivity. We use several synthetic and field seismic data to illustrate the effectiveness of the presented method. [ABSTRACT FROM AUTHOR]

The Barzilai–Borwein (BB) step size initially proposed in the context of unconstrained optimization has become one of the most popular step choices for gradient-based methods in the optimization community. It is well-known that the powerful variational inequality can be used to characterize the first-order optimality condition of constrained optimization problems. However, a variational inequality problem is not always equivalent to a constrained optimization problem. In this paper, we follow the spirit of BB step size and propose a projection method with alternate BB step size for general variational inequalities. Although the global convergence is established under some strong conditions, a series of computational experiments on nonlinear complementarity problems, image deblurring problems and generalized Nash equilibrium problems demonstrate that the proposed almost-parameter-free projection method is more efficient than some existing state-of-the-art projection methods in the literature. [ABSTRACT FROM AUTHOR]

We describe an algorithm based on a logarithmic barrier function, Newton's method and linear conjugate gradients that seeks an approximate minimizer of a smooth function over the non-negative orthant. We develop a bound on the complexity of the approach, stated in terms of the required accuracy and the cost of a single gradient evaluation of the objective function and/or a matrix-vector multiplication involving the Hessian of the objective. The approach can be implemented without explicit calculation or storage of the Hessian. [ABSTRACT FROM AUTHOR]

We propose a new stepsize for the gradient method. It is shown that this new stepsize will converge to the reciprocal of the largest eigenvalue of the Hessian, when Dai-Yang's asymptotic optimal gradient method (Computational Optimization and Applications, 2006, 33(1): 73–88) is applied for minimizing quadratic objective functions. Based on this spectral property, we develop a monotone gradient method that takes a certain number of steps using the asymptotically optimal stepsize by Dai and Yang, and then follows by some short steps associated with this new stepsize. By employing one step retard of the asymptotic optimal stepsize, a nonmonotone variant of this method is also proposed. Under mild conditions, R-linear convergence of the proposed methods is established for minimizing quadratic functions. In addition, by combining gradient projection techniques and adaptive nonmonotone line search, we further extend those methods for general bound constrained optimization. Two variants of gradient projection methods combining with the Barzilai-Borwein stepsizes are also proposed. Our numerical experiments on both quadratic and bound constrained optimization indicate that the new proposed strategies and methods are very effective. [ABSTRACT FROM AUTHOR]

We propose a derivative-free algorithm for solving linear equality constrained non-linear optimization problems, named LECOA. In each iteration of the algorithm, the objective function is approximated by a quadratic model constructed from interpolation points. The choice of the points leaves some degree of freedom in the model taken up by minimizing the Frobenius norm of the change to the Hessian matrix of the model. The new iterate is generally generated by minimizing the model in a trust-region using a null space truncated conjugate gradient method. Numerical results are presented which show that the proposed algorithm competes against some algorithms in the literature. Experiments show that starting with the point that minimizes the infinity norm subject to the linear equality constraints gives excellent results. A limit-type global convergence of the proposed algorithm is proved under some reasonable assumptions. [ABSTRACT FROM AUTHOR]

We study an inverse problem associated with an eddy current model. We first address the ill-posedness of the inverse problem by proving the compactness of the forward map with respect to the conductivity and the nonuniqueness of the recovery process. Then by virtue of nonradiating source conceptions, we establish a regularity result for the tangential trace of the true solution on the boundary, which is necessary to justify our subsequent mathematical formulation. After that, we formulate the inverse problem as a constrained optimization problem with an appropriate regularization and prove the existence and stability of the regularized minimizers. To facilitate the numerical solution of the nonlinear nonconvex constrained optimization, we introduce a feasible Lagrangian and its discrete variant. Then the gradient of the objective functional is derived using the adjoint technique. By means of the gradient, a nonlinear conjugate gradient method is formulated for solving the optimization system, and a Sobolev gradient is incorporated to accelerate the iterative process. Numerical examples are provided to demonstrate the feasibility of the proposed algorithm. [ABSTRACT FROM AUTHOR]

The first‐order reliability method (FORM) is well recognized as an efficient approach for reliability analysis. Rooted in considering the reliability problem as a constrained optimization of a function, the traditional FORM makes use of gradient‐based optimization techniques to solve it. However, the gradient‐based optimization techniques may result in local convergence or even divergence for the highly nonlinear or high‐dimensional performance function. In this paper, a hybrid method combining the Salp Swarm Algorithm (SSA) and FORM is presented. In the proposed method, a Lagrangian objective function is constructed by the exterior penalty function method to facilitate meta‐heuristic optimization strategies. Then, SSA with strong global optimization ability for highly nonlinear and high‐dimensional problems is utilized to solve the Lagrangian objective function. In this regard, the proposed SSA‐FORM is able to overcome the limitations of FORM including local convergence and divergence. Finally, the accuracy and efficiency of the proposed SSA‐FORM are compared with two gradient‐based FORMs and several heuristic‐based FORMs through eight numerical examples. The results show that the proposed SSA‐FORM can be generally applied for reliability analysis involving low‐dimensional, high‐dimensional, and implicit performance functions. [ABSTRACT FROM AUTHOR]

In this work, we tend to deal within the field of the constrained optimization methods of three-term Conjugate Gradient (CG) technique which is primarily based on Dai–Liao (DL) formula. The new proposed technique satisfies the conjugacy property and the descent conditions of Karush–Kuhn–Tucker (K.K.T.). Our planned constrained technique uses the robust Wolfe line search condition with some assumptions. We tend to prove the global convergence property of the new planned technique. Numeral comparisons for (30-thirty) constrained optimization issues make sure the effectiveness of the new planned formula. [ABSTRACT FROM AUTHOR]

We present a novel fully adaptive conditional gradient method with the step length regulation for solving pseudo-convex constrained optimization problems. We propose some deterministic rules of the step length regulation in a normalized direction. These rules guarantee to find the step length by utilizing the finite procedures and provide the strict relaxation of the objective function at each iteration. We prove that the sequence of the function values for the iterates generated by the algorithm converges globally to the objective function optimal value with sublinear rate. [ABSTRACT FROM AUTHOR]

Motivated by the applications of the node-based shape optimization problems, where various response evaluations are often considered in constrained optimization, we propose a modified search direction method for optimization design updates. There exist numerous methods for solving constrained optimization problems. The methods of the class active-set strategy try to travel along the active constraints to solve constrained optimization problems. Contrary to the active-set methods, the algorithms of the class interior-point method reach an optimal solution by traversing the interior of the feasible domain. This characteristic is considered to be beneficial for shape optimization because the usual zig-zagging behavior when traveling along the active constraints is avoided. However, the interior-point methods require the solution of a Newton problem in each iteration step, which is generally considered to be difficult for the shape optimization problem. In the present work, we propose a modified search direction method that applies the singular-value decomposition method for the normalized objective and constraint gradients. The modified search direction is a descent direction of the objective function. Using this search direction for the design update in each iteration, we observe that the centrality conditions for the interior-point method are approached iteratively. We are able to achieve a local minimum by traversing the interior of the feasible domain by only using the gradient information of the objective and constraint functions. The results are shown first with analytical 2D problems, and then the results of shape optimization problems with a large number of design variables are discussed. To robustly deal with complex geometries, the Vertex Morphing method is used. [ABSTRACT FROM AUTHOR]

Abstract The role of the steplength selection strategies in gradient methods has been widely investigated in the last decades. Starting from the work of Barzilai and Borwein (1988), many efficient steplength rules have been designed, that contributed to make the gradient approaches an effective tool for the large-scale optimization problems arising in important real-world applications. Most of these steplength rules have been thought in unconstrained optimization, with the aim of exploiting some second-order information for achieving a fast annihilation of the gradient of the objective function. However, these rules are successfully used also within gradient projection methods for constrained optimization, though, to our knowledge, a detailed analysis of the effects of the constraints on the steplength selections is still not available. In this work we investigate how the presence of the box constraints affects the spectral properties of the Barzilai–Borwein rules in quadratic programming problems. The proposed analysis suggests the introduction of new steplength selection strategies specifically designed for taking account of the active constraints at each iteration. The results of a set of numerical experiments show the effectiveness of the new rules with respect to other state of the art steplength selections and their potential usefulness also in case of box-constrained non-quadratic optimization problems. [ABSTRACT FROM AUTHOR]

The objective of this paper is to look for structural designs arising from topological optimization procedures that aim at maximizing the loading capacity regarding incipient plastic collapse. The mechanical problem is described by limit analysis (LA) formulation that allows a direct determination of the load that produces the plastic collapse phenomenon without information about the load history. In case of proportional loading processes, LA consists of computing a critical load factor such that the structure undergoes plastic collapse when the reference load is amplified by this factor. In this case, LA can be cast mathematically as a convex constrained optimization problem. The design optimization is formally stated as the maximization of the collapse load factor subject to a fixed quantity of available material. The design is controlled by solid isotropic microstructure with penalization (SIMP) technique. In the particular case of the chosen objective function, the solution of the adjoint problem in sensitivity analysis coincides with the Newton–Raphson update vector obtained at the convergence of the procedure developed to solve the LA optimization problem, fact that reduces the numerical cost of gradient calculations. In order to keep the implementation straightforward, the optimality conditions are solved by a classical heuristic element-by-element density updating algorithm, well known in the literature. The set of tested examples brings encouraging results with structures being stressed to ultimate bearing states. Implementation was kept as simple as possible, leaving the field open to further investigations. Numerical tests show that, despite having similar geometries, plastic collapse factor obtained with compliance optimal designs are lower than those obtained with present formulation. [ABSTRACT FROM AUTHOR]

Abstract The conjugate gradient methods (CGMs) are very effective iterative methods for solving large-scale unconstrained optimization. The aim of this work is to improve the Fletcher–Reeves and Dai–Yuan CGMs. First, based on the conjugate parameters of the Fletcher–Reeves (FR) method and the Dai–Yuan (DY) method, and combining the second inequality of the strong Wolfe line search, two new conjugate parameters are constructed. Second, using the two new conjugate parameters, another FR type conjugate parameter is presented. Third, utilizing the strong Wolfe line search to yield the steplength, three improved CGMs are proposed for large-scale unconstrained optimization. Under usual assumptions, the improved methods are all proved to possess sufficient descent property and global convergence. Finally, three group experiments and their corresponding performance profiles are reported, which show that the proposed methods are very promising. [ABSTRACT FROM AUTHOR]

In this work, we propose nonlinear conjugate gradient methods for finding critical points of vector-valued functions with respect to the partial order induced by a closed, convex, and pointed cone with nonempty interior. No convexity assumption is made on the objectives. The concepts of Wolfe and Zoutendjik conditions are extended for the vector-valued optimization. In particular, we show that there exist intervals of step sizes satisfying the Wolfe-type conditions. The convergence analysis covers the vector extensions of the Fletcher{Reeves, conjugate descent, Dai-Yuan, Polak-Ribière-Polyak, and Hestenes-Stiefel parameters that retrieve the classical ones in the scalar minimization case. Under inexact line searches and without regular restarts, we prove that the sequences generated by the proposed methods find points that satisfy the first-order necessary condition for Pareto-optimality. Numerical experiments illustrating the practical behavior of the methods are presented. [ABSTRACT FROM AUTHOR]

Multiple shooting methods for solving optimal control problems have been developed rapidly in the past decades and are widely considered a promising direction to speed up the optimization process. Here we propose and analyze a new multiple shooting algorithm based on a sequential quadratic programming (SQP) method that is suitable for optimal control problems governed by large-scale time-dependent partial-differential equations (PDEs). We investigate the structure of the KKT matrix and solve the large-scale KKT system by a preconditioned conjugate gradient algorithm. A simplified block Schur complement preconditioner is proposed, that allows for the parallelization of the method in the time domain. The proposed algorithm is first validated for an optimal control problem constrained by the Nagumo equation. The results indicate that considerable accelerations can be achieved for multiple shooting approaches with appropriate starting guesses and scaling of the matching conditions. We further apply the proposed algorithm to a two-dimensional velocity tracking problem governed by the Navier–Stokes equations. We find algorithmic speed-ups of up to 12 versus single shooting on up to 50 shooting windows. We also compare results with earlier work that uses an augmented Lagrangian algorithm instead of SQP, showing better performance of the SQP method for most of the cases. • Novel multiple shooting algorithm for PDE optimal control problems. • Use of an SQP method for solving the equality constrained optimization. • Preconditioned conjugate gradient algorithm for solving the KKT system. • Use of a simplified block Schur complement preconditioner. • Significant algorithmic speed-ups for up to 100 shooting windows. [ABSTRACT FROM AUTHOR]

We present a competitive algorithm to minimize a locally Lipschitz function constrained with locally Lipschitz constraints. The approach is to use an ℓ nonsmooth penalty function. The method generates second order descent directions to minimize the ℓ penalty function. We introduce a new criterion to decide upon acceptability of a Goldstein subdifferential approximation. We show that the new criterion leads to an improvement of the Goldstein subdifferential approximation, as introduced by Mahdavi-Amiri and Yousefpour. Also, making use of our proposed line search strategy, the method always moves on differentiable points. Furthermore, the method has an adaptive behaviour in the sense that, when the iterates move on adequately smooth regions, the search directions switch exactly to the Shanno's conjugate gradient directions and no subdifferential approximation is computed. The global convergence of the algorithm is established. Finally, an extensive comparison of the results obtained by our proposed algorithm with the ones obtained by some well-known methods shows significant reductions in number of function and gradient evaluations. [ABSTRACT FROM AUTHOR]

In this paper, we consider a multivariate spectral DY-type projection method for solving nonlinear monotone equations with convex constraints. The search direction of the proposed method combines those of the multivariate spectral gradient method and DY conjugate gradient method. With no need for the derivative information, the proposed method is very suitable to solve large-scale nonsmooth monotone equations. Under appropriate conditions, we prove the global convergence and R-linear convergence rate of the proposed method. The preliminary numerical results also indicate that the proposed method is robust and effective. [ABSTRACT FROM AUTHOR]

In this paper, we focus on the stochastic inverse eigenvalue problem of reconstructing a stochastic matrix from the prescribed spectrum. We directly reformulate the stochastic inverse eigenvalue problem as a constrained optimization problem over several matrix manifolds to minimize the distance between isospectral matrices and stochastic matrices. Then we propose a geometric Polak--Ribiere--Polyak-based nonlinear conjugate gradient method for solving the constrained optimization problem. The global convergence of the proposed method is established. Our method can also be extended to the stochastic inverse eigenvalue problem with prescribed entries. An extra advantage is that our models yield new isospectral flow methods. Finally, we report some numerical tests to illustrate the efficiency of the proposed method for solving the stochastic inverse eigenvalue problem and the case of prescribed entries. [ABSTRACT FROM AUTHOR]

Conjugate gradient methods are an important class of methods for unconstrained optimization, especially for large-scale problems. Recently, they have been much studied. In this paper, a new family of conjugate gradient method is proposed for unconstrained optimization. This method includes the already existing two practical nonlinear conjugate gradient methods, which produces a descent search direction at every iteration and converges globally provided that the line search satisfies the Wolfe conditions. The numerical experiments are done to test the efficiency of the new method, which implies the new method is promising. In addition the methods related to this family are uniformly discussed. [ABSTRACT FROM AUTHOR]

In this paper, we propose a new conjugate gradient (CG) method which belongs to the CG methods of Dai–Liao family [New conjugacy conditions and related nonlinear conjugate gradient methods, Appl. Math. Optim. 43 (2001), pp. 87–101]. Babaie-Kafaki et al. [Two new conjugate gradient methods based on modified secant equations, J. Comput. Appl. Math. 234 (2010), pp. 1374–1386] made some modifications on the Yabe and Takano's CG approach [Global convergence properties of nonlinear conjugate gradient methods with modified secant condition, Comput. Optim. Appl. 28 (2004), pp. 203–225] and received some appealing results in theory and practice. Here, we introduce an efficient updating rule for the parameters of the Yabe and Takano's CG algorithm. Under some standard assumptions, we establish the global convergence property of the new suggested algorithm on uniformly convex and general functions. Numerical results on some testing problems from CUTEr collection show the priority of the proposed method to some existing CG methods in practice. [ABSTRACT FROM AUTHOR]

Nonlinear conjugate gradient method is very popular in solving large-scale unconstrained minimization problems due to its simple iterative form and lower storage requirement. In the recent years, it was successfully extended to solve higher-dimension monotone nonlinear equations. Nevertheless, the research activities on conjugate gradient method in symmetric equations are just beginning. This study aims to developing, analyzing, and validating a family of nonlinear conjugate gradient methods for symmetric equations. The proposed algorithms are based on the latest, and state-of-the-art descent conjugate gradient methods for unconstrained minimization. The series of proposed methods are derivative-free, where the Jacobian information is needless at the full iteration process. We prove that the proposed methods converge globally under some appropriate conditions. Numerical results with differentiable parameter's values and performance comparisons with another solver CGD to demonstrate the superiority and effectiveness of the proposed algorithms are reported. [ABSTRACT FROM AUTHOR]

In this note, we show that there are some improvements in the proof of Theorem 2 and Theorem 4 in Andrei (Numer. Algor. 47, 143-156, 2008). Moreover, the assumption 0 < c≤ 휃 < 1 of Theorem 2 can be deleted. Throughout, we use the same notations and equation numbers as Andrei (Numer. Algor. 47, 143-156, 2008). [ABSTRACT FROM AUTHOR]

In this paper, a self-adjust conjugate gradient method is proposed for solving unconstrained problems, which can generate sufficient descent directions at each iteration. Different from the existent methods, a dynamical adjustment of conjugacy condition in our proposed method is developed, which can be regarded as the inheritance and development of properties of standard Hestenes-Stiefel method. Under mild condition, we show the proposed method convergent globally even if the objective function is nonconvex. Numerical results illustrate that our method can efficiently solve the test problems and therefore is promising. [ABSTRACT FROM AUTHOR]

The introduction of quasi-Newton and nonlinear conjugate gradient methods revolutionized the field of nonlinear optimization. The self-scaling memoryless Broyden-Fletcher-Goldfarb-Shanno (SSML-BFGS) method by Perry (Disscussion Paper 269, ) and Shanno (SIAM J Numer Anal, 15, 1247-1257, ) provided a good understanding about the relationship between the two classes of methods. Based on the SSML-BFGS method, new conjugate gradient algorithms, called CG_DESCENT and CGOPT, have been proposed by Hager and Zhang (SIAM J Optim, 16, 170-192, ) and Dai and Kou (SIAM J Optim, 23, 296-320, ), respectively. It is somewhat surprising that the two conjugate gradient methods perform more efficiently than the SSML-BFGS method. In this paper, we aim at proposing some suitable modifications of the SSML-BFGS method such that the sufficient descent condition holds. Convergence analysis of the modified method is made for convex and nonconvex functions, respectively. The numerical experiments for the testing problems from the Constrained and Unconstrained Test Environment collection demonstrate that the modified SSML-BFGS method yields a desirable improvement over CGOPT and the original SSML-BFGS method. [ABSTRACT FROM AUTHOR]

Recently, conjugate gradient methods, which usually generate descent search directions, are useful for large-scale optimization. Narushima et al. (SIAM J Optim 21:212-230, ) have proposed a three-term conjugate gradient method which satisfies a sufficient descent condition. We extend this method to two parameters family of three-term conjugate gradient methods which can be used to control the magnitude of the directional derivative. We show that these methods converge globally and work well for suitable choices of the parameters. Numerical results are also presented. [ABSTRACT FROM AUTHOR]