Search

Optimization methods are widely used in various fields to find the best possible solution to a given problem through the minimization or maximization of an objective function while adhering to specific constraints. In this paper, we present a new optimization algorithm, called WOADD, which was designed to handle the challenges of constrained optimization problems that involve dependent data. Unlike traditional algorithms that struggle with data dependencies and valid range constraints, WOADD uses a unique normalization process and a dynamic updating mechanism that accurately considers the interdependencies among features. By calculating a scaling parameter to navigate within feasible regions, WOADD adjusts its search strategy to ensure the preservation of data dependencies and adherence to constraints, ultimately leading to more efficient and precise optimization outcomes. Our extensive experimental analysis, which compared WOADD against other swarm optimization methods using a suite of benchmark functions, demonstrates its superior performance in terms of faster convergence rates, improved solution quality, and enhanced determinism in outcomes. [ABSTRACT FROM AUTHOR]

We consider a sequential quadratic programming algorithm for optimization problems with equality and inequality constraints, equipped with the standard Armijo linesearch procedure for a nonsmooth exact penalty function, intended for globalization of convergence. We are interested in the case when the standard assumptions for local superlinear convergence of the method may not hold. Specifically, we allow for violation of standard constraint qualifications and second-order sufficient optimality conditions, in which case attraction to so-called critical Lagrange multipliers is known to have a negative impact on convergence rate. In these circumstances, some known acceleration techniques can be expected to take effect only provided the true Hessian and the full SQP step are asymptotically accepted, and these are the main issues addressed in this work. The presented constructions extend some previously known ones to the case when inequality constraints are involved. [ABSTRACT FROM AUTHOR]

This paper proposes a posteriori error control for the discretization of variational inequalities with linear constraints in an abstract framework. The central aspect is the discussion of the error contributions representing the non-penetration, non-conformity and complementarity conditions, which are typically given by some cut-off functions. Replacing the standard cut-off functions with the minimizers of a weighted functional enables the derivation of reliable and, in particular, efficient a posteriori error estimates. The abstract findings are applied to the obstacle problem as well as to the simplified Signorini problem, where higher-order finite elements are used to provide appropriate discretization spaces. Numerical experiments show that the error estimates based on this new approach have (nearly) constant efficiency indeces and reflect the expected order of convergence when uniform h-refinements are applied. Moreover, they can be used to steer adaptive schemes in order to improve the order of convergence. The numerical results are compared with estimates resulting from the standard cut-off functions. [ABSTRACT FROM AUTHOR]

Abstract In this paper, a nonconforming finite element method (NFEM) is proposed for the constrained optimal control problems (OCPs) governed by a bilinear state equation. The state and adjoint state are approximated by the nonconforming E Q 1 r o t element, and the control is approximated by the orthogonal projection through the state and adjoint state. Some superclose and superconvergence properties are obtained by full use of the distinguish characters of this E Q 1 r o t element, such as the interpolation operator equals the Ritz projection, and the consistency error is one order higher than its interpolation error in the broken energy norm. Finally, some numerical results are provided to verify the theoretical analysis. [ABSTRACT FROM AUTHOR]

We consider the problem of minimizing a continuous function that may be non-smooth and non-convex, subject to bound constraints. We propose an algorithm that uses the L-BFGS quasi-Newton approximation of the problem's curvature together with a variant of the weak Wolfe line search. The key ingredient of the method is an active-set selection strategy that defines the subspace in which search directions are computed. To overcome the inherent shortsightedness of the gradient for a non-smooth function, we propose two strategies. The first relies on an approximation of the ε-minimum norm subgradient, and the second uses an iterative corrective loop that augments the active set based on the resulting search directions. While theoretical convergence guarantees have been elusive even for the unconstrained case, we present numerical results on a set of standard test problems to illustrate the efficacy of our approach, using an open-source Python implementation of the proposed algorithm. [ABSTRACT FROM AUTHOR]

We study the problem of trajectory planning for autonomous vehicles designed to minimize the travel distance while avoiding moving obstacles whose position and speed are not known. Because, in practice, observations from sensors have measurement errors, the stochasticity of the data is modeled using maximum likelihood estimators, which are shown to be consistent as the sample size increases. To comply with the kinematic constraints of the vehicle, we consider smooth trajectories that can be represented by a linear combination of B-spline basis functions, transforming the infinite-dimensional problem into a finite-dimensional one. Moreover, a smooth penalty function is added to the travel distance, transforming the constrained optimization problem into an unconstrained one. The planned stochastic trajectory, obtained from the minimization problem with stochastic confidence regions, is shown to converge almost surely to the deterministic one as the number of sensor observations increases. Finally, we present two simulation studies to demonstrate the proposed method. [ABSTRACT FROM AUTHOR]

For constrained equations with nonisolated solutions, we show that if the equation mapping is 2-regular at a given solution with respect to a direction in the null space of the Jacobian, and this direction is interior feasible, then there is an associated domain of starting points from which a family of Newton-type methods is well defined and necessarily converges to this specific solution (despite degeneracy, and despite that there are other solutions nearby). We note that unlike the common settings of convergence analyses, our assumptions subsume that a local Lipschitzian error bound does not hold for the solution in question. Our results apply to constrained and projected variants of the Gauss-Newton, Levenberg-Marquardt, and LP-Newton methods. Applications to smooth and piecewise smooth reformulations of complementarity problems are also discussed. [ABSTRACT FROM AUTHOR]

Despite the lack of theoretical and practical convergence support, the Nelder-Mead (NM) algorithm is widely used to solve unconstrained optimization problems. It is a derivative-free algorithm, that attempts iteratively to replace the worst point of a simplex by a better one. The present paper proposes a way to extend the NM algorithm to inequality constrained optimization. This is done through a search step of the mesh adaptive direct search (Mads) algorithm, inspired by the NM algorithm. The proposed algorithm does not suffer from the NM lack of convergence, but instead inherits from the totality of the Mads convergence analysis. Numerical experiments show an important improvement in the quality of the solutions produced using this search step. [ABSTRACT FROM AUTHOR]

In this work, we propose nonlinear conjugate gradient methods for finding critical points of vector-valued functions with respect to the partial order induced by a closed, convex, and pointed cone with nonempty interior. No convexity assumption is made on the objectives. The concepts of Wolfe and Zoutendjik conditions are extended for the vector-valued optimization. In particular, we show that there exist intervals of step sizes satisfying the Wolfe-type conditions. The convergence analysis covers the vector extensions of the Fletcher{Reeves, conjugate descent, Dai-Yuan, Polak-Ribière-Polyak, and Hestenes-Stiefel parameters that retrieve the classical ones in the scalar minimization case. Under inexact line searches and without regular restarts, we prove that the sequences generated by the proposed methods find points that satisfy the first-order necessary condition for Pareto-optimality. Numerical experiments illustrating the practical behavior of the methods are presented. [ABSTRACT FROM AUTHOR]

The scaled gradient projection (SGP) method, which can be viewed as a promising improvement of the classical gradient projection method, is a quite efficient solver for real-world problems arising in image science and machine learning. Most recently, Bonettini and Prato [Inverse Probl. 2015;31:095008. 20 p] proved that the SGP method with the monotone Armijo line search technique has the convergence rate, where k counts the iteration. In this paper, we first show that the SGP method could be equipped with the nonmonotone line search procedure proposed by Zhang and Hager [SIAM J Optim. 2004;14:1043-1056]. To some extent, such a nonmonotone technique might improve the performance of SGP method, because its effectiveness has been verified for unconstrained optimization by comparing with the traditional monotone and nonmonotone strategies. Then, we prove that the new SGP method also has the convergence rate under the condition that the objective function is convex. Furthermore, we derive the linear convergence of the SGP algorithm under the strongly convexity assumption of the involved objective function. [ABSTRACT FROM AUTHOR]

Highlights • The balance of convergence and diversity are analyzed and divided into horizontal imbalance and vertical imbalance. • The one-to-one correspondence between the region and the subproblem is established by region division. • Adaptive Region Adjustment (ARA) strategy is proposed to improve the balance between convergence and diversity. • A comprehensive experiment is designed on UF and MOP to prove the effectiveness of our proposed algorithm. Abstract The multiobjective evolutionary algorithm based on decomposition (MOEA/D), which decomposes a multiobjective optimization problem (MOP) into a number of optimization subproblems and optimizes them in a collaborative manner, becomes more and more popular in the field of evolutionary multiobjective optimization. The mechanism of balance convergence and diversity is very important in MOEA/D. In the process of optimization, the chosen solutions must be distinctive and as close as possible to the Pareto front. In this paper, we first explore the relation between subproblems and solutions. Then we propose the adaptive region adjustment strategy to balance the convergence and diversity based on the objective region partition concept. Finally, this strategy is embedded in the MOEA/D framework and then a simple but efficient algorithm is proposed. To demonstrate the effectiveness of the proposed algorithm, comprehensive experiments have been designed. The simulation results show the effectiveness of our proposed algorithms. [ABSTRACT FROM AUTHOR]

Graphical abstract Highlights • A modified MOPSO is proposed by implementing new dynamic boundary search procedure. • It is tested on constrained benchmark, biochemical and tumor treatment problems. • The results are favourable compared to other metaheuristics used in this paper. Abstract Due to increased search complexity in multi-objective optimization, premature convergence becomes a problem. Complex engineering problems poses high number of variables with many constraints. Hence, more difficult benchmark problems must be utilized to validate new algorithms performance. A well-known optimizer, Multi-Objective Particle Swarm Optimizer (MOPSO), has a few weakness that needs to be addressed, specifically its convergence in high dimensional problems and its constraints handling capability. For these reasons, we propose a modified MOPSO (M-MOPSO) to improve upon these aspects. M-MOPSO is compared with four other algorithms namely, MOPSO, Multi-Objective Grey Wolf Optimizer (MOGWO), Multi-Objective Evolutionary Algorithm based on Decompositions (MOEA/D) and Multi-Objective Differential Evolution (MODE). M-MOPSO emerged as the best algorithm in eight out of the ten constrained benchmark problems. It also shows promising results in bioprocess application problems and tumor treatment problems. In overall, M-MOPSO was able to solve multi-objective problems with good convergence and is suitable to be used in real world problem. [ABSTRACT FROM AUTHOR]

Approximation is central to many optimization problems and the supporting theory provides insight as well as foundation for algorithms. In this paper, we lay out a broad framework for quantifying approximations by viewing finite- and infinite-dimensional constrained minimization problems as instances of extended real-valued lower semicontinuous functions defined on a general metric space. Since the Attouch-Wets distance between such functions quantifies epi-convergence, we are able to obtain estimates of optimal solutions and optimal values through bounds of that distance. In particular, we show that near-optimal and near-feasible solutions are effectively Lipschitz continuous with modulus one in this distance. Under additional assumptions on the underlying metric space, we construct approximating functions involving only a finite number of parameters that still are close to an arbitrary extended real-valued lower semicontinuous functions. [ABSTRACT FROM AUTHOR]

The performance of branch-and-bound algorithms for deterministic global optimization is strongly dependent on the ability to construct tight and rapidly convergent schemes of lower bounds. One metric of the efficiency of a branch-and-bound algorithm is the convergence order of its bounding scheme. This article develops a notion of convergence order for lower bounding schemes for constrained problems, and defines the convergence order of convex relaxation-based and Lagrangian dual-based lower bounding schemes. It is shown that full-space convex relaxation-based lower bounding schemes can achieve first-order convergence under mild assumptions. Furthermore, such schemes can achieve second-order convergence at KKT points, at Slater points, and at infeasible points when second-order pointwise convergent schemes of relaxations are used. Lagrangian dual-based full-space lower bounding schemes are shown to have at least as high a convergence order as convex relaxation-based full-space lower bounding schemes. Additionally, it is shown that Lagrangian dual-based full-space lower bounding schemes achieve first-order convergence even when the dual problem is not solved to optimality. The convergence order of some widely-applicable reduced-space lower bounding schemes is also analyzed, and it is shown that such schemes can achieve first-order convergence under suitable assumptions. Furthermore, such schemes can achieve second-order convergence at KKT points, at unconstrained points in the reduced-space, and at infeasible points under suitable assumptions when the problem exhibits a specific separable structure. The importance of constraint propagation techniques in boosting the convergence order of reduced-space lower bounding schemes (and helping mitigate clustering in the process) for problems which do not possess such a structure is demonstrated. [ABSTRACT FROM AUTHOR]

In this paper, we propose two feasible methods based on projections using a curvilinear search for solving optimization problems with orthogonality constraints. In one of them we apply a projected Adams-Moulton-like update scheme. All our algorithms compute the SVD decomposition in each iteration to preserve feasibility. Additionally, we present some convergence results. Finally, we perform numerical experiments with simulated problems; and analyze the performance of the proposed methods compared with state-of-the-art algorithms. [ABSTRACT FROM AUTHOR]

This paper proposes a multi-objective version of recently developed Spotted Hyena Optimizer (SHO) called Multi-objective Spotted Hyena Optimizer (MOSHO). It is used to optimize the multiple objectives problems. In the proposed algorithm, a fixed-sized archive is employed for storing the non-dominated Pareto optimal solutions. The roulette wheel mechanism is used to select the effective solutions from archive to simulate the social and hunting behaviors of spotted hyenas. The proposed algorithm is tested on 24 benchmark test functions and compared with six recently developed metaheuristic algorithms. The proposed algorithm is then applied on six constrained engineering design problems to demonstrate its applicability on real-life problems. The experimental results reveal that the proposed algorithm performs better than the others and produces the Pareto optimal solutions with high convergence. [ABSTRACT FROM AUTHOR]

We propose a method for equality-constrained optimization based on a problem in which all constraints are systematically regularized. The regularization is equivalent to applying an augmented Lagrangian method but the linear system used to compute a search direction is reminiscent of regularized sequential quadratic programming. A limited-memory BFGS approximation to second derivatives allows us to employ iterative methods for linear least squares to compute steps, resulting in a factorization-free implementation. We establish global and fast local convergence under weak assumptions. In particular, we do not require the LICQ and our method is suitable for degenerate problems. Preliminary numerical experiments show that a factorization-based implementation of our method exhibits significant robustness while a factorization-free implementation, though not as robust, is promising. We briey discuss generalizing our framework to other classes of methods and to problems with inequality constraints. [ABSTRACT FROM AUTHOR]

We present a family of new inexact secant methods in association with Armijo line search technique for solving nonconvex constrained optimization. Different from the existing inexact secant methods, the algorithms proposed in this paper need not compute exact directions. By adopting the nonsmooth exact penalty function as the merit function, the global convergence of the proposed algorithms is established under some reasonable conditions. Some numerical results indicate that the proposed algorithms are both feasible and effective. [ABSTRACT FROM AUTHOR]

We discuss several optimization procedures to solve finite element approximations of linear-quadratic Dirichlet optimal control problems governed by an elliptic partial differential equation posed on a 2D or 3D Lipschitz domain. The control is discretized explicitly using continuous piecewise linear approximations. Unconstrained, control-constrained, state-constrained and control-and-state constrained problems are analysed. A preconditioned conjugate method for a reduced problem in the control variable is proposed to solve the unconstrained problem, whereas semismooth Newton methods are discussed for the solution of constrained problems. State constraints are treated via a Moreau-Yosida penalization. Convergence is studied for both the continuous problems and the finite dimensional approximations. In the finite dimensional case, we are able to show convergence of the optimization procedures even in the absence of Tikhonov regularization parameter. Computational aspects are also treated and several numerical examples are included to illustrate the theoretical results. [ABSTRACT FROM AUTHOR]

The generalized Nash equilibrium problem (GNEP) is often difficult to solve by Newton-type methods since the problem tends to have locally nonunique solutions. Here we take an existing trust-region method which is known to be locally fast convergent under a relatively mild error bound condition, and modify this method by a nonmonotone strategy in order to obtain a more reliable and efficient solver. The nonmonotone trust-region method inherits the nice local convergence properties of its monotone counterpart and is also shown to have the same global convergence properties. Numerical results indicate that the nonmonotone trust-region method is significantly better than the monotone version, and is at least competitive to an existing software applied to the same reformulation used within our trust-region framework. Additional tests on quasi-variational inequalities (QVI) are also presented to validate efficiency of the proposed extension. [ABSTRACT FROM AUTHOR]

We propose a new globalization strategy that can be used in unconstrained optimization algorithms to support rapid convergence from remote starting points. Our approach is based on using multiple points at each iteration to build a sequence of representative models of the objective function. Using the new information gathered from those multiple points, a local step is gradually improved by updating its direction as well as its length. We give a global convergence result and also provide the parallel implementation details accompanied with a numerical study. Our numerical study shows that the proposed algorithm is a promising alternative as a globalization strategy. [ABSTRACT FROM PUBLISHER]

Applications in the areas of data fitting, forecasting, and estimation naturally lead to a rich class of constrained infinite-dimensional optimization problems over extended real-valued semicontinuous functions. We discuss a framework for dealing with such applications, even in the presence of nearly arbitrary constraints on the functions. We formulate computationally tractable approximating problems relying on piecewise polynomial semicontinuous approximations of the actual functions. The approximations enable the study of evolving problems caused by incrementally arriving data and other information. In contrast to an earlier more general treatment, we focus on optimization over functions defined on a compact interval of the real line, which still addresses a large number of applications. The paper provides an introduction to the subject through simplified derivations and illustrative examples. [ABSTRACT FROM AUTHOR]

In this paper, we present a QP-free algorithm without a penalty function or a filter for nonlinear general-constrained optimization. At each iteration, three systems of linear equations with the same coefficient matrix are solved to yield search direction; the nonmonotone line search ensures that the objective function or constraint violation function is sufficiently reduced. There is no feasibility restoration phase in our algorithm, which is necessary for filter methods. The algorithm possesses global convergence as well as superlinear convergence under some mild conditions including a weaker assumption of positive definiteness. Finally, some preliminary numerical results are reported. [ABSTRACT FROM AUTHOR]

The standard BFGS method plays an important role among the quasi-Newton algorithms for constrained/un-constrained optimization problems. However, Dai (2003) constructed a counterexample to demonstrate that this method may fail for non-convex functions with inexact Wolfe line searches, and Mascarenhas (2004) showed the nonconvergence of this method and other methods in the Broyden family, even with exact line search techniques. These works motivate us to try to find another way to obtain another quasi-Newton method with better convergence based on the standard BFGS formula. In this paper, four approaches are used in the designed algorithm: (1) a modified weak Wolfe–Powell line search technique is introduced; (2) if one defined condition is satisfied, then the search direction and its associated step length are accepted, and the next point is designed; (3) otherwise, a parabolic will be presented, and it is regarded as the projection surface to avoid using the failed direction, and the next point x k + 1 is generated by a projection technique; and (4) to easily obtain the global convergence of the proposed algorithm, the projection point is not used in the current BFGS update but rather for all the next iterations. The new algorithm possesses global convergence for general functions under the inexact modified weak Wolfe–Powell line search technique, and it is shown that other methods in the Broyden class also have this property. Numerical results are reported to show the performance of the given algorithm and other similar methods. [ABSTRACT FROM AUTHOR]

When solving constrained multi-objective optimization problems (CMOPs), keeping infeasible individuals with good objective values and small constraint violations in the population can improve the performance of the algorithms, since they provide the information about the optimal direction towards Pareto front. By taking the constraint violation as an objective, we propose a novel constraint-handling technique based on directed weights to deal with CMOPs. This paper adopts two types of weights, i.e. feasible and infeasible weights distributing on feasible and infeasible regions respectively, to guide the search to the promising region. To utilize the useful information contained in infeasible individuals, this paper uses infeasible weights to maintain a number of well-diversified infeasible individuals. Meanwhile, they are dynamically changed along with the evolution to prefer infeasible individuals with better objective values and smaller constraint violations. Furthermore, 18 test instances and 2 engineering design problems are used to evaluate the effectiveness of the proposed algorithm. Several numerical experiments indicate that the proposed algorithm outperforms four compared algorithms in terms of finding a set of well-distributed non-domination solutions. [ABSTRACT FROM AUTHOR]

In this article, A novel nature-inspired optimization algorithm known as Lightning Attachment Procedure Optimization (LAPO) is proposed. The proposed approach mimics the lightning attachment procedure including the downward leader movement, the upward leader propagation, the unpredictable trajectory of lightning downward leader, and the branch fading feature of lightning. Final optimum result would be the lightning striking point. The proposed method is free from any parameter tuning and it is rarely stuck in the local optimum points. To evaluate the proposed algorithm, 29 mathematical benchmark functions are employed and the results are compared to those of 9 high quality well-known optimization methods The results of the proposed method are compared from different points of views, including quality of the results, convergence behavior, robustness, and CPU time consumption. Superiority and high quality performance of the proposed method are demonstrated through comparing the results. Moreover, the proposed method is also tested by five classical engineering design problems including tension/compression spring, welded beam, pressure vessel designs, Gear train design, and Cantilever beam design and a high constraint optimization problem known as Optimal Power Flow (OPF) which is a high constraint electrical engineering problem. The excellence performance of the proposed method in solving the problems with large number of constraints and also discrete optimization problems are also concluded from the results of the six engineering problem. [ABSTRACT FROM AUTHOR]

In the recent literature a popular algorithm namely ‘Competitive Swarm Optimizer (CSO)’ has been proposed for solving unconstrained optimization problems that updates only half of the population in each iteration. A modified CSO (MCSO) is being proposed in this paper where two thirds of the population swarms are being updated by a tri-competitive criterion unlike CSO. A small change in CSO makes a huge difference in the solution quality. The basic idea behind the proposition is to maintain a higher rate of exploration to the search space with a faster rate of convergence. The proposed MCSO is applied to solve the standard CEC2008 and CEC2013 large scale unconstrained benchmark optimization problems. The empirical results and statistical analysis confirm the better overall performance of MCSO over many other state-of-the-art meta-heuristics, including CSO. In order to confirm the superiority further, a real life problem namely ‘sampling-based image matting problem’ is solved. Considering the winners of CEC 2008 and 2013, MCSO attains the second best position in the competition. [ABSTRACT FROM AUTHOR]

This paper proposes a self-adaptive penalty function and presents a penalty-based algorithm for solving nonsmooth and nonconvex constrained optimization problems. We prove that the general constrained optimization problem is equivalent to a bound constrained problem in the sense that they have the same global solutions. The global minimizer of the penalty function subject to a set of bound constraints may be obtained by a population-based meta-heuristic. Further, a hybrid self-adaptive penalty firefly algorithm, with a local intensification search, is designed, and its convergence analysis is established. The numerical experiments and a comparison with other penalty-based approaches show the effectiveness of the new self-adaptive penalty algorithm in solving constrained global optimization problems. [ABSTRACT FROM AUTHOR]

Nonmonotone spectral gradient (NSG) techniques are considered for unconstrained optimization of differentiable functions. They combine a nonmonotone step length strategy, that is based on the Grippo-Lampariello-Lucidi nonmonotone line search, with the spectral gradient choice of steplength. In this paper, we present a hybrid nonmonotone spectral gradient method. In each iteration after finding an acceptable point by NSG method, we employ an extrapolation step to find a even better point by using particle swarm optimization. Through specific assumptions, we prove the global convergence of our proposed algorithm [ABSTRACT FROM AUTHOR]

In this paper, a modified BFGS algorithm is proposed for unconstrained optimization. The proposed algorithm has the following properties: (i) a nonmonotone line search technique is used to obtain the step size $\alpha_{k}$ to improve the effectiveness of the algorithm; (ii) the algorithm possesses not only global convergence but also superlinear convergence for generally convex functions; (iii) the algorithm produces better numerical results than those of the normal BFGS method. [ABSTRACT FROM AUTHOR]

In this paper, a novel hybrid lightning search algorithm-simplex method (LSA-SM) is proposed to solve the shortcomings of lightning search algorithm (LSA) premature convergence and low computational accuracy and it is applied to function optimization and constrained engineering design optimization problems. The improvement adds two major optimization strategies. Simplex method (SM) iteratively optimizes the current worst step leaders to avoid the population searching at the edge, thus improving the convergence accuracy and rate of the algorithm. Elite opposition-based learning (EOBL) increases the diversity of population to avoid the algorithm falling into local optimum. LSA-SM is tested by 18 benchmark functions and five constrained engineering design problems. The results show that LSA-SM has higher computational accuracy, faster convergence rate, and stronger stability than other algorithms and can effectively solve the problem of constrained nonlinear optimization in reality. [ABSTRACT FROM AUTHOR]

This paper reviews recent studies on the Particle Swarm Optimization (PSO) algorithm. The review has been focused on high impact recent articles that have analyzed and/or modified PSO algorithms. This paper also presents some potential areas for future study. [ABSTRACT FROM AUTHOR]

A modified variant of artificial bee colony (ABC) algorithms, called radial artificial bee colony (RABC) algorithm, was proposed. This modified method incorporated two novel strategies during the initialization of employed bees and the determination of new locations for scout bees. RABC was applied to solve well-known constrained and unconstrained problems. The statistical results of RABC were compared with those of the original ABC and a number of approaches from the past studies. The comparisons revealed that RABC was superior to its competitor in terms of accuracy, speed of convergence and consistency. [ABSTRACT FROM AUTHOR]

This note serves two purposes. Firstly, we construct a counterexample to show that the statement on the convergence of the alternating direction method of multipliers (ADMM) for solving linearly constrained convex optimization problems in a highly influential paper by Boyd et al. (Found Trends Mach Learn 3(1):1-122, 2011) can be false if no prior condition on the existence of solutions to all the subproblems involved is assumed to hold. Secondly, we present fairly mild conditions to guarantee the existence of solutions to all the subproblems of the ADMM and provide a rigorous convergence analysis on the ADMM with a computationally more attractive large step-length that can even exceed the practically much preferred golden ratio of $$(1+\sqrt{5})/2$$ . [ABSTRACT FROM AUTHOR]

We present a primal–dual augmented Lagrangian method to solve an equality constrained minimization problem. This is a Newton-like method applied to a perturbation of the optimality system that follows from a reformulation of the initial problem by introducing an augmented Lagrangian function. An important aspect of this approach is that, by a choice of suitable updating rules of parameters, the algorithm reduces to a regularized Newton method applied to a sequence of optimality systems. The global convergence is proved under mild assumptions. An asymptotic analysis is also presented and quadratic convergence is proved under standard regularity assumptions. Some numerical results show that the method is very efficient and robust. [ABSTRACT FROM AUTHOR]

We consider the problem of minimizing a given n-variate polynomial f over the hypercube [-1, 1]n. An idea introduced by Lasserre, is to find a probability distribution on [-1, 1]n with polynomial density function h (of given degree r) that minimizes the expectation ∩[-1 1]n∩(x)h(x)dμ(x), where dμ(x) is a fixed, finite Borel measure supported on [- 1, 1]n. It is known that, for the Lebesgue measure dμ(x) = dx, one may show an error bound O(1/√r) if h is a sum-of-squares density, and an O(1/r) error bound if h is the density of a beta distribution. In this paper, we show an error bound of O(1/r²), if dμ(x) = (∏in=1 √1-xi²) (the well-known measure in the study of orthogonal polynomials), and h has a Schmiidgen-type representation with respect to [-1,1]n, which is a more general condition than a sum of squares. The convergence rate analysis relies on the theory of polynomial kernels and, in particular, on Jackson kernels. We also show that the resulting upper bounds may be computed as generalized eig envalue problems, as is also the case for sum-of-squares densities. [ABSTRACT FROM AUTHOR]

How to solve constrained optimization problems constitutes an important part of the research on optimization problems. In this paper, a hybrid immune clonal particle swarm optimization multi-objective algorithm is proposed to solve constrained optimization problems. In the proposed algorithm, the population is first initialized with the theory of good point set. Then, differential evolution is adopted to improve the local optimal solution of each particle, with immune clonal strategy incorporated to improve each particle. As a final step, sub-swarm is used to enhance the position and velocity of individual particle. The new algorithm has been tested on 24 standard test functions and three engineering optimization problems, whose results show that the new algorithm has good performance in both robustness and convergence. [ABSTRACT FROM AUTHOR]

The optimization-based stabilization of manifolds for nonlinear dynamical systems with constraints is investigated. Manifolds in the state or output space are considered for which the original system description can be transformed into a so-called transverse normal form. With this formulation, the motion of the system transverse and tangential to the manifold can be separately described. The transverse normal form is combined with a tailored model predictive control scheme to achieve the objectives of stabilizing the manifold, rendering it invariant, and imposing a desired motion on the manifold under due consideration of constraints. Furthermore, the stabilization of the manifold is prioritized over the movement on the manifold. Convergence of the model predictive control scheme is proven. The applicability of the proposed concept is demonstrated by an illustrative simulation example. [ABSTRACT FROM AUTHOR]

This paper proposes a practical formulation for the non-convex economic dispatch problem to consider multi-fuel options, ramp rate limits, valve loading effect, prohibited operating zones and spinning reserve. A new optimization algorithm based on the θ -bat algorithm ( θ -BA) is suggested to solve the problem. The θ -BA converts the Cartesian search space into the polar coordinates such that more search ability would be achieved. According to the complex, nonlinear, and constrained nature of the problem, a new self-adaptive modification method is proposed. The proposed modified θ -BA ( θ -MBA) is constructed based on the roulette wheel mechanism to effectively increase the convergence of the algorithm. The high ability and satisfying performance of the proposed optimization method is examined on IEEE 15-unit, 40-unit and 100-unit test systems. [ABSTRACT FROM AUTHOR]

In this paper, we present a flexible nonmonotone filter method for solving nonlinear constrained optimization problems which are common models in industry. This new method has more flexibility for the acceptance of the trial step compared to the traditional filter methods, and requires less computational costs compared with the monotone-type methods. Moreover, we use a self-adaptive parameter to adjust the acceptance criteria, so that Maratos effect can be avoided a certain degree. Under reasonable assumptions, the proposed algorithm is globally convergent. Numerical tests are presented that confirm the efficiency of the approach. [ABSTRACT FROM AUTHOR]

Chen and Zhang [ Sci. China, Ser. A, 45, 1390-1397 (2002)] introduced an affine scaling trust region algorithm for linearly constrained optimization and analyzed its global convergence. In this paper, we derive a new affine scaling trust region algorithm with dwindling filter for linearly constrained optimization. Different from Chen and Zhang's work, the trial points generated by the new algorithm are accepted if they improve the objective function or improve the first order necessary optimality conditions. Under mild conditions, we discuss both the global and local convergence of the new algorithm. Preliminary numerical results are reported. [ABSTRACT FROM AUTHOR]

By means of Wolfe conditions strategy, we propose a quasi-Newton trust-region method to solve box constrained optimization problems. This method is an adequate combination of the compact limited memory BFGS and the trust-region direction while the generated point satisfies the Wolfe conditions and therefore maintains a positive-definite approximation to the Hessian of the objective function. The global convergence and the quadratic convergence of this method are established under suitable conditions. Finally, we compare our algorithms (IWTRAL and IBWTRAL) with an active set trust-region algorithm (ASTRAL) Xu and Burke (2007) on the CUTEst box constrained test problems presented by Gould et al. (2015). Numerical results show that the presented method is competitive and totally interesting for solving box constrained optimization. [ABSTRACT FROM AUTHOR]

The aim of this paper is to present the convergence analysis of a very general class of gradient projection methods for smooth, constrained, possibly nonconvex, optimization. The key features of these methods are the Armijo linesearch along a suitable descent direction and the non Euclidean metric employed to compute the gradient projection. We develop a very general framework from the point of view of block-coordinate descent methods, which are useful when the constraints are separable. In our numerical experiments we consider a large scale image restoration problem to illustrate the impact of the metric choice on the practical performances of the corresponding algorithm. [ABSTRACT FROM AUTHOR]

This study concerns with a trust-region-based method for solving unconstrained optimization problems. The approach takes the advantages of the compact limited memory BFGS updating formula together with an appropriate adaptive radius strategy. In our approach, the adaptive technique leads us to decrease the number of subproblems solving, while utilizing the structure of limited memory quasi-Newton formulas helps to handle large-scale problems. Theoretical analysis indicates that the new approach preserves the global convergence to a fi rst-order stationary point under classical assumptions. Moreover, the superlinear and the quadratic convergence rates are also established under suitable conditions. Preliminary numerical experiments show the effectiveness of the proposed approach for solving large-scale unconstrained optimization problems. [ABSTRACT FROM AUTHOR]

This paper develops a particle swarm optimization (PSO) based framework for constrained optimization problems (COPs). Aiming at enhancing the performance of PSO, a modified PSO algorithm, named SASPSO 2011, is proposed by adding a newly developed self-adaptive strategy to the standard particle swarm optimization 2011 (SPSO 2011) algorithm. Since the convergence of PSO is of great importance and significantly influences the performance of PSO, this paper first theoretically investigates the convergence of SASPSO 2011. Then, a parameter selection principle guaranteeing the convergence of SASPSO 2011 is provided. Subsequently, a SASPSO 2011-based framework is established to solve COPs. Attempting to increase the diversity of solutions and decrease optimization difficulties, the adaptive relaxation method, which is combined with the feasibility-based rule, is applied to handle constraints of COPs and evaluate candidate solutions in the developed framework. Finally, the proposed method is verified through 4 benchmark test functions and 2 real-world engineering problems against six PSO variants and some well-known methods proposed in the literature. Simulation results confirm that the proposed method is highly competitive in terms of the solution quality and can be considered as a vital alternative to solve COPs. [ABSTRACT FROM AUTHOR]

The exact penalty methods are very popular because of their ability to handle degenerate problems and inconsistent constraint linearizations. This paper presents a line search exact penalty method with bi-object strategy (LSBO) for nonlinear constrained optimization. In the algorithm LSBO, the penalty parameter is selected at every iteration such that the sufficient progress toward feasibility and optimality is guaranteed along the search direction. In contrast with classical exact penalization approaches, LSBO method has two goals to determine whether the current iteration is successful or not. One is improving the feasibility and the other is reducing the value of the objective function. Moreover, the penalty parameter is only related to the information at the current iterate point. The sequence of the penalty parameter is non-monotone, which does not affect the global convergence in theory and is seen to be advantageous in practice. It is shown that the algorithm enjoys favorable global convergence properties under the weaker assumptions. Numerical experiments illustrate the behavior of the algorithm on various difficult situations. [ABSTRACT FROM AUTHOR]

We propose a novel actor–critic algorithm with guaranteed convergence to an optimal policy for a discounted reward Markov decision process. The actor incorporates a descent direction that is motivated by the solution of a certain non-linear optimization problem. We also discuss an extension to incorporate function approximation and demonstrate the practicality of our algorithms on a network routing application. [ABSTRACT FROM AUTHOR]