Search

Direct Multisearch (DMS) is a well-established class of methods for multiobjective derivative-free optimization, where constraints are addressed by an extreme barrier approach, only evaluating feasible points. In this work, we propose the replacement of this extreme barrier approach by a filter strategy, combined with an inexact feasibility restoration step, to address constraints in the DMS framework. The filter approach treats feasibility as an additional component of the objective function that needs to be minimized. The inexact restoration step attempts to generate new feasible points, contributing to prioritize this feasibility, a requirement for the good performance of any filter approach. Theoretical results are provided, analysing the different types of sequences of points generated by the new algorithm, and numerical experiments on a set of nonlinearly constrained biobjective problems are reported, stating the good algorithmic performance of the proposed approach. [ABSTRACT FROM AUTHOR]

In this paper, a numerical approximation method is developed to find approximate solutions to a class of constrained multi-objective optimization problems. All the functions of the problem are not necessarily convex functions. At each iteration of the method, a particular type of subproblem is solved using the trust region technique, and the step is evaluated using the notions of actual reduction and predicted reduction. A non-differentiable $ l_{\infty } $ l∞ penalty function restricts the constraint violations. An adaptive BFGS update formula is introduced. Global convergence of the proposed algorithm is established under the Mangasarian-Fromovitz constraint qualification and some mild assumptions. Furthermore, it is justified that the proposed algorithm displays a super-linear convergence rate. Numerical results are provided to show the efficiency of the algorithm in the quality of the approximated Pareto front. [ABSTRACT FROM AUTHOR]

We propose a novel steplength selection rule in proximal gradient methods for minimizing the sum of a differentiable function plus an $ \def\bigtimes{{\Huge \times}} \def\operatornamewithlimits#1{\mathop{\rm #1}\limits} \ell _1 $ ℓ 1 -norm penalty term. The proposed rule modifies one of the classical Barzilai–Borwein steplength, extending analogous results obtained in the context of gradient projection methods for constrained optimization. We analyse the spectral properties of the Barzilai–Borwein-like steplength when the differentiable part is quadratic, showing that its reciprocal lies in the spectrum of the submatrix of the Hessian that depends on both the nonzero and the nonoptimal zero components of the current iterate, allowing for acceleration effects when the optimal zero components start to be identified. Furthermore, we insert the modified rule into a proximal gradient method with a nonmonotone line search, for which we prove global convergence towards a stationary point. Numerical experiments show the ability of the proposed rule to sweep the spectrum of the reduced Hessian on a series of quadratic $ \ell _1 $ ℓ 1 -regularized problems, as well as its effectiveness in recovering the ground truth in a least squares regularized problem arising in image restoration. [ABSTRACT FROM AUTHOR]

In this paper, we propose a spectral projected subgradient method with a 1-memory momentum term for solving constrained convex multiobjective optimization problem. This method combines the subgradient-type algorithm for multiobjective optimization problems with the idea of the spectral projected algorithm to accelerate the convergence process. Additionally, a 1-memory momentum term is added to the subgradient direction in the early iterations. The 1-memory momentum term incorporates, in the present iteration, some of the influence of the past iterations, and this can help to improve the search direction. Under suitable assumptions, we show that the sequence generated by the method converges to a weakly Pareto efficient solution and derive the sublinear convergence rates for the proposed method. Finally, computational experiments are given to demonstrate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

This paper proposes a novel Difference-of-Convex (DC) decomposition for polynomials using a power-sum representation, achieved by solving a sparse linear system. We introduce the Boosted DCA with Exact Line Search ( BDCA e ) for addressing linearly constrained polynomial programs within the DC framework. Notably, we demonstrate that the exact line search equates to determining the roots of a univariate polynomial in an interval, with coefficients being computed explicitly based on the power-sum DC decompositions. The subsequential convergence of BDCA e to critical points is proven, and its convergence rate under the Kurdyka–Łojasiewicz property is established. To efficiently tackle the convex subproblems, we integrate the Fast Dual Proximal Gradient method by exploiting the separable block structure of the power-sum DC decompositions. We validate our approach through numerical experiments on the Mean–Variance–Skewness–Kurtosis portfolio optimization model and box-constrained polynomial optimization problems. Comparative analysis of BDCA e against DCA, BDCA with Armijo line search, UDCA, and UBDCA with projective DC decomposition, alongside standard nonlinear optimization solvers FMINCON and FILTERSD, substantiates the efficiency of our proposed approach. [ABSTRACT FROM AUTHOR]

In the smooth constrained optimization setting, this work introduces the Domain Complementary Approximate Karush–Kuhn–Tucker (DCAKKT) condition, inspired by a sequential optimality condition recently devised for non-smooth constrained optimization problems. It is shown that the augmented Lagrangian method can generate limit points satisfying DCAKKT, and it is proved that such a condition is not related to previously established sequential optimality conditions. An essential characteristic of the DCAKKT is to capture the asymptotic potential increasing of the Lagrange multipliers using a single parameter. Besides that, DCAKKT points satisfy the Strong Approximate Gradient Projection (SAGP) condition. Due to the intrinsic features of DCAKKT, which combine strength and generality, this novel and genuine sequential optimality condition may shed some light upon the practical performance of algorithms that are yet to be devised. [ABSTRACT FROM AUTHOR]

In this two-part study, we develop a general theory of the so-called exact augmented Lagrangians for constrained optimization problems in Hilbert spaces. In contrast to traditional nonsmooth exact penalty functions, these augmented Lagrangians are continuously differentiable for smooth problems and do not suffer from the Maratos effect, which makes them especially appealing for applications in numerical optimization. Our aim is to present a detailed study of various theoretical properties of exact augmented Lagrangians and discuss several applications of these functions to constrained variational problems, problems with PDE constraints, and optimal control problems. The first paper is devoted to a theoretical analysis of an exact augmented Lagrangian for optimization problems in Hilbert spaces. We obtain several useful estimates of this augmented Lagrangian and its gradient, and present several types of sufficient conditions for KKT-points of a constrained problem corresponding to locally/globally optimal solutions to be local/global minimizers of the exact augmented Lagrangian. [ABSTRACT FROM AUTHOR]

We present the Branch-and-Bound Performance Estimation Programming (BnB-PEP), a unified methodology for constructing optimal first-order methods for convex and nonconvex optimization. BnB-PEP poses the problem of finding the optimal optimization method as a nonconvex but practically tractable quadratically constrained quadratic optimization problem and solves it to certifiable global optimality using a customized branch-and-bound algorithm. By directly confronting the nonconvexity, BnB-PEP offers significantly more flexibility and removes the many limitations of the prior methodologies. Our customized branch-and-bound algorithm, through exploiting specific problem structures, outperforms the latest off-the-shelf implementations by orders of magnitude, accelerating the solution time from hours to seconds and weeks to minutes. We apply BnB-PEP to several setups for which the prior methodologies do not apply and obtain methods with bounds that improve upon prior state-of-the-art results. Finally, we use the BnB-PEP methodology to find proofs with potential function structures, thereby systematically generating analytical convergence proofs. [ABSTRACT FROM AUTHOR]

In this paper, by revisiting intersection rules for normal cones, we give formulas for estimating or computing the Fréchet/Mordukhovich/Moreau–Rockafellar subdifferentials of optimal value functions of constrained parametric optimization problems under metric qualification conditions. The results are then applied to derive chain rules for composite functions in both convex and nonconvex situations. Illustrative examples and comparisons to existing results, including those of Mordukhovich and Shao (Trans Amer Math Soc 348:1235–1280, 1996), Mordukhovich et al. (Math Program Ser B 116:369–396, 2009) and of An and Jourani (J Optim Theory Appl 192:82–96, 2022), are also addressed. [ABSTRACT FROM AUTHOR]

Optimization problems with least constraint violation have many important practical backgrounds. The related works in the literature normally focus on the norm square measure function and the classical augmented Lagrangian method is proved to be suitable for finding an approximate solution. This paper considers the constrained convex optimization problem with the least constraint violation under a general measure function. The properties of the conjugate dual associated with the measure function of the shifted problem are discussed through the relations between the dual function and the optimal value function. The differentiability of the dual function associated with the measure function is proved. The properties of augmented Lagrangian method induced by the measure function are characterized in terms of the dual function. The optimality conditions for the problem with the least constraint violation are established in term of the augmented Lagrangian. It is shown that the augmented Lagrangian method has the properties that the sequence of shift measure values is decreasing, the sequence of multipliers is unbounded, and the sequence of shifts converges to the least violated shift under an extra assumption. [ABSTRACT FROM AUTHOR]

We investigate finite-dimensional constrained structured optimization problems, featuring composite objective functions and set-membership constraints. Offering an expressive yet simple language, this problem class provides a modeling framework for a variety of applications. We study stationarity and regularity concepts, and propose a flexible augmented Lagrangian scheme. We provide a theoretical characterization of the algorithm and its asymptotic properties, deriving convergence results for fully nonconvex problems. It is demonstrated how the inner subproblems can be solved by off-the-shelf proximal methods, notwithstanding the possibility to adopt any solvers, insofar as they return approximate stationary points. Finally, we describe our matrix-free implementation of the proposed algorithm and test it numerically. Illustrative examples show the versatility of constrained composite programs as a modeling tool and expose difficulties arising in this vast problem class. [ABSTRACT FROM AUTHOR]

There are many important practical optimization problems whose feasible regions are not known to be nonempty or not, and optimizers of the objective function with the least constraint violation prefer to be found. A natural way for dealing with these problems is to extend the nonlinear optimization problem as the one optimizing the objective function over the set of points with the least constraint violation. This leads to the study of the shifted problem. This paper focuses on the constrained convex optimization problem. The sufficient condition for the closedness of the set of feasible shifts is presented and the continuity properties of the optimal value function and the solution mapping for the shifted problem are studied. Properties of the conjugate dual of the shifted problem are discussed through the relations between the dual function and the optimal value function. The solvability of the dual of the optimization problem with the least constraint violation is investigated. It is shown that, if the least violated shift is in the domain of the subdifferential of the optimal value function, then this dual problem has an unbounded solution set. Under this condition, the optimality conditions for the problem with the least constraint violation are established in term of the augmented Lagrangian. It is shown that the augmented Lagrangian method has the properties that the sequence of shifts converges to the least violated shift and the sequence of multipliers is unbounded. Moreover, it is proved that the augmented Lagrangian method is able to find an approximate solution to the problem with the least constraint violation and it has linear rate of convergence under an error bound condition. The augmented Lagrangian method is applied to an illustrative convex second-order cone constrained optimization problem with least constraint violation and numerical results verify our theoretical results. [ABSTRACT FROM AUTHOR]

In this paper, an efficient parallel splitting method is proposed for the optimal control problem with parabolic equation constraints. The linear finite element is used to approximate the state variable and the control variable in spatial direction. And the Crank–Nicolson scheme is applied to discretize the constraint equation in temporal direction. For consistency, the trapezoidal rule and midpoint rule are used to approximate the integrals with respect to the state variable and the control variable of the objective functional in temporal direction, respectively. Based on the separable structure of the resulting coupled discretized optimization system, a full Jacobian decomposition method with correction is adopted to solve the decoupled subsystems in parallel, which improves the computational efficiency significantly. Moreover, the global convergence estimate is established using the discretization error by the finite element and the iteration error by the full Jacobian decomposition method with correction. Finally, numerical simulations are carried out to verify the efficiency of the proposed method. [ABSTRACT FROM AUTHOR]

Linear hyperspectral unmixing (LHU) is a class of important problems in remote sensing. It can be modelled by a linearly constrained convex optimization problem with a coupled objective function. This paper proposes an alternating direction method of multipliers (ADMM) for solving this LHU model. The special structure of the LHU model allows explicit solutions to the subproblems in the ADMM and hence the ADMM is easily implementable. The global convergence of the ADMM is established despite the existence of a coupled term in the objective function. Our numerical experiments with four data sets demonstrated that the proposed ADMM is effective for solving the LHU model. [ABSTRACT FROM AUTHOR]

Focusing on smooth constrained optimization problems, and inspired by the complementary approximate Karush–Kuhn–Tucker (CAKKT) conditions, this work introduces the weighted complementary approximate Karush–Kuhn–Tucker (WCAKKT) conditions. They are shown to be verified by limit points generated not only by safeguarded augmented Lagrangian methods, but also by inexact restoration methods, inverse and logarithmic barrier methods, and a penalized algorithm for constrained nonsmooth optimization. Under the analyticity of the feasible set description, and resting upon a desingularization result, the new conditions are proved to be equivalent to the CAKKT conditions. The WCAKKT conditions capture the algebraic elements of the desingularization result needed to characterize CAKKT sequences using a weighted complementarity condition that asymptotically sums zero. Due to its generality and strength, the new condition may help to enlighten the practical performance of algorithms in generating CAKKT sequences. [ABSTRACT FROM AUTHOR]

In this paper, we consider a class of constrained optimization problems whose constraints involve a cardinality or rank constraint. The penalty formulation based on a partial regularization has recently been promoted in the literature to approximate these problems, which usually outperforms the penalty formulation based on a full regularization in terms of solution quality. Nevertheless, the relation between the penalty formulation with a partial regularizer and the original problem was not much studied yet. Under some suitable assumptions, we show that the penalty formulation based on a partial regularization is an exact reformulation of the original problem in the sense that they both share the same global minimizers. We also show that a local minimizer of the original problem is that of the penalty reformulation. These results provide some theoretical justification for the often-observed superior performance of the penalty model based on a partial regularizer over a corresponding full regularizer. [ABSTRACT FROM AUTHOR]

The goal of this paper is to investigate an approach for derivative-free optimization that has not received sufficient attention in the literature and is yet one of the simplest to implement and parallelize. In its simplest form, it consists of employing derivative-based methods for unconstrained or constrained optimization and replacing the gradient of the objective (and constraints) by finite-difference approximations. This approach is applicable to problems with or without noise in the functions. The differencing interval is determined by a bound on the second (or third) derivative and by the noise level, which is assumed to be known or to be accessible through difference tables or sampling. The use of finite-difference gradient approximations has been largely dismissed in the derivative-free optimization literature as too expensive in terms of function evaluations or as impractical in the presence of noise. However, the test results presented in this paper suggest that it has much to be recommended. The experiments compare newuoa, dfo-ls and cobyla against finite-difference versions of l-bfgs, lmder and knitro on three classes of problems: general unconstrained problems, nonlinear least squares problems and nonlinear programs with inequality constraints. [ABSTRACT FROM AUTHOR]

The adaptive regularisation algorithm using cubics (ARC) is initially proposed for unconstrained optimization. ARC has excellent convergence properties and complexity. In this paper, we extend ARC to solve nonlinear equality constrained optimization and propose a sequential adaptive regularisation using cubics algorithm inspired by sequential quadratic programming (SQP) methods. In each iteration of our method, the trial step is computed via composite-step approach, i.e., it is decomposed into the sum of normal step and tangential step. By means of reduced-Hessian approach, a new ARC subproblem for nonlinear equality constrained optimization is constructed to compute the tangential step, which can supply sufficient decrease required in the proposed algorithm. Once the trial step is obtained, the ratio of the penalty function reduction to the model function reduction is calculated to determine whether the trial point is accepted. The global convergence of the algorithm is investigated under some mild assumptions. Preliminary numerical experiments are reported to show the performance of the proposed algorithm. [ABSTRACT FROM AUTHOR]

In this paper, we consider the continuous relaxation reformulation of cardinality-constrained optimization problems that has become more popular in recent years and propose a new sequential optimality condition (approximate stationarity) for cardinality-constrained optimization problems, which is proved to be a genuine necessary optimality condition that does not require any constraint qualification to hold. We compare this condition with the rest of the sequential optimality conditions and prove that our condition is stronger and closer to the local minimizer. A problem-tailored regularity condition is proposed, and we show that this regularity condition ensures that the approximate stationary point proposed in this paper is the exact stationary point and is the weakest constraint qualification with this property. Finally, we apply the results of this paper to safeguarded augmented Lagrangian method and prove that the algorithm converges to the approximate stationary point proposed in this paper under mild assumptions, the existing theoretical results of this algorithm are further enhanced. [ABSTRACT FROM AUTHOR]

We consider the bound-constrained global optimization of functions with low effective dimensionality, that are constant along an (unknown) linear subspace and only vary over the effective (complement) subspace. We aim to implicitly explore the intrinsic low dimensionality of the constrained landscape using feasible random embeddings, in order to understand and improve the scalability of algorithms for the global optimization of these special-structure problems. A reduced subproblem formulation is investigated that solves the original problem over a random low-dimensional subspace subject to affine constraints, so as to preserve feasibility with respect to the given domain. Under reasonable assumptions, we show that the probability that the reduced problem is successful in solving the original, full-dimensional problem is positive. Furthermore, in the case when the objective's effective subspace is aligned with the coordinate axes, we provide an asymptotic bound on this success probability that captures its polynomial dependence on the effective and, surprisingly, ambient dimensions. We then propose X-REGO, a generic algorithmic framework that uses multiple random embeddings, solving the above reduced problem repeatedly, approximately and possibly, adaptively. Using the success probability of the reduced subproblems, we prove that X-REGO converges globally, with probability one, and linearly in the number of embeddings, to an ϵ -neighbourhood of a constrained global minimizer. Our numerical experiments on special structure functions illustrate our theoretical findings and the improved scalability of X-REGO variants when coupled with state-of-the-art global—and even local—optimization solvers for the subproblems. [ABSTRACT FROM AUTHOR]

This paper aims to develop distributed algorithms for nonconvex optimization problems with complicated constraints associated with a network. The network can be a physical one, such as an electric power network, where the constraints are nonlinear power flow equations, or an abstract one that represents constraint couplings between decision variables of different agents. Despite the recent development of distributed algorithms for nonconvex programs, highly complicated constraints still pose a significant challenge in theory and practice. We first identify some difficulties with the existing algorithms based on the alternating direction method of multipliers (ADMM) for dealing with such problems. We then propose a reformulation that enables us to design a two-level algorithm, which embeds a specially structured three-block ADMM at the inner level in an augmented Lagrangian method framework. Furthermore, we prove the global and local convergence as well as iteration complexity of this new scheme for general nonconvex constrained programs, and show that our analysis can be extended to handle more complicated multi-block inner-level problems. Finally, we demonstrate with computation that the new scheme provides convergent and parallelizable algorithms for various nonconvex applications, and is able to complement the performance of the state-of-the-art distributed algorithms in practice by achieving either faster convergence in optimality gap or in feasibility or both. [ABSTRACT FROM AUTHOR]

A gradient projection method with feasible inexact projections is proposed in the present paper. The inexact projection is performed using a general relative error tolerance. Asymptotic convergence analysis under quasiconvexity assumption and iteration-complexity bounds under convexity assumption of the method employing constant and Armijo step sizes are presented. Numerical results are reported illustrating the potential advantages of considering inexact projections instead of exact ones in some medium scale instances of a least squares problem over the spectrohedron. [ABSTRACT FROM AUTHOR]

Asymptotic stationarity and regularity conditions turned out to be quite useful to study the qualitative properties of numerical solution methods for standard nonlinear and complementarity constrained programs. In this paper, we first extend these notions to nonlinear optimization problems with nonsmooth but Lipschitzian data functions in order to find reasonable notions of asymptotic stationarity and regularity in terms of Clarke's and Mordukhovich's subdifferential construction. Particularly, we compare the associated novel asymptotic constraint qualifications with already existing ones. The second part of the paper presents two applications of the obtained theory. On the one hand, we specify our findings for complementarity constrained optimization problems and recover recent results from the literature which demonstrates the power of the approach. Furthermore, we hint at potential extensions to or- and vanishing constrained optimization. On the other hand, we demonstrate the usefulness of asymptotic regularity in the context of bilevel optimization. More precisely, we justify a well-known stationarity system for affinely constrained bilevel optimization problems in a novel way. Afterwards, we suggest a solution algorithm for this class of bilevel optimization problems which combines a penalty method with ideas from DC-programming. After a brief convergence analysis, we present the results of some numerical experiments. [ABSTRACT FROM AUTHOR]

Gradient projection methods represent effective tools for solving large-scale constrained optimization problems thanks to their simple implementation and low computational cost per iteration. Despite these good properties, a slow convergence rate can affect gradient projection schemes, especially when high accurate solutions are needed. A strategy to mitigate this drawback consists in properly selecting the values for the steplength along the negative gradient. In this paper, we consider the class of gradient projection methods with line search along the projected arc for box-constrained minimization problems and we analyse different strategies to define the steplength. It is well known in the literature that steplength selection rules able to approximate, at each iteration, the eigenvalues of the inverse of a suitable submatrix of the Hessian of the objective function can improve the performance of gradient projection methods. In this perspective, we propose an automatic hybrid steplength selection technique that employs a proper alternation of standard Barzilai–Borwein rules, when the final active set is not well approximated, and a generalized limited memory strategy based on the Ritz-like values of the Hessian matrix restricted to the inactive constraints, when the final active set is reached. Numerical experiments on quadratic and non-quadratic test problems show the effectiveness of the proposed steplength scheme. [ABSTRACT FROM AUTHOR]

In this manuscript, we consider multi-objective optimization problems with a cardinality constraint on the vector of decision variables and additional linear constraints. For this class of problems, we analyse necessary and sufficient conditions of Pareto optimality. We afterwards propose a Penalty Decomposition type algorithm, exploiting multi-objective descent methods, to tackle the aforementioned family of problems. We conduct a rigorous convergence analysis for the proposed method, where we prove that the produced sequence of points has limit points, each one being feasible and satisfying first-order optimality conditions. Numerical computational experiments, carried out on instances of relevant real-world problems such as sparse mean/variance portfolio selection and sparse regularized logistic regression, in their multi-objective formulation, show that the proposed procedure is effective at finding solutions forming good Pareto sets approximations. [ABSTRACT FROM AUTHOR]

Semi-infinite programming can be used to model a large variety of complex optimization problems. The simple description of such problems comes at a price: semi-infinite problems are often harder to solve than finite nonlinear problems. In this paper, we combine a classical adaptive discretization method developed by Blankenship and Falk [Infinitely constrained optimization problems. J Opt Theory Appl. 1976;19(2):261–281. https://doi.org/10.1007/BF00934096] and techniques regarding a semi-infinite optimization problem as a bi-level optimization problem. We develop a new adaptive discretization method which combines the advantages of both techniques and exhibits a quadratic rate of convergence. We further show that a limit of the iterates is a stationary point, if the iterates are stationary points of the approximate problems. [ABSTRACT FROM AUTHOR]

This paper proposes a globally convergent regularized interior point method that involves a specifically designed regularization strategy for constrained optimization. The main concept of the proposed algorithm is that when a proper regularization parameter is selected, the direction obtained from the regularized barrier equation is a descent direction for either the objective function or constraint violation. Accordingly, by embedding a flexible strategy of choosing a regularization parameter in a trust-funnel-like interior point scheme, we propose the new algorithm. Global convergence under the mild assumptions of relaxed constant rank constraint qualification (RCRCQ) and local consistency of the linearized active and equality constraints is shown. Preliminary numerical experiments are conducted, and the results are encouraging. [ABSTRACT FROM AUTHOR]

In this paper, we consider the weak approximate solutions to constrained vector optimization problems with variable ordering structures. In terms of abstract subdifferentials, normal cones and coderivatives, we establish Lagrange rules for this kind of solutions. [ABSTRACT FROM AUTHOR]

In this paper, we consider nonlinear optimization problems with nonlinear equality constraints and bound constraints on the variables. For the solution of such problems, many augmented Lagrangian methods have been defined in the literature. Here, we propose to modify one of these algorithms, namely ALGENCAN by Andreani et al., in such a way to incorporate second-order information into the augmented Lagrangian framework, using an active-set strategy. We show that the overall algorithm has the same convergence properties as ALGENCAN and an asymptotic quadratic convergence rate under suitable assumptions. The numerical results confirm that the proposed algorithm is a viable alternative to ALGENCAN with greater robustness. [ABSTRACT FROM AUTHOR]

In this article, a class of lower-order smoothed objective penalty functions is introduced to find locally optimal points for constrained optimization problems. The exactness of the new penalty functions is studied. Based on the current locally optimal points, a new class of penalty functions based on filling properties is proposed. This new penalty function can be used to find a better locally optimal point. The exactness and filling properties of this penalty function are proved in this paper. To do this, two algorithms are presented to find the locally and globally optimal points. Additionally, their convergence is proved under some mild conditions. Finally, numerical results are included to illustrate the applicability of the local and global optimization algorithms. [ABSTRACT FROM AUTHOR]

In this paper, an adaptation of the sequential quadratically constrained quadratic programming method is proposed to solve inequality constrained minimization. At each iteration, it solves a norm-relaxed quadratically constrained quadratic programming subproblem that uses an active set identification technique to reduce the scale and computational cost. By taking a valid line search, the iterates always get into the feasible set after a finite number of iterations, and followed by a suitable update rule for the penalty parameters. Under suitable conditions without the strict complementarity, our method has the global and superlinear convergence properties. In addition, numerical results are reported to demonstrate the efficiency of the proposed method. [ABSTRACT FROM AUTHOR]

Canonical duality theory (CDT) is presented by its creator DY Gao as a theory which can be used for solving a large class of challenging real-world problems. It is the aim of this paper to study rigorously constrained optimization problems in finite dimensional spaces using the method suggested by CDT and to discuss several results published in the last ten years. [ABSTRACT FROM AUTHOR]

We consider a projected gradient method equipped with the nonmonotone line search procedure for convex constrained multiobjective optimization problems. Under mild assumptions, we show the convergence of the full sequence generated by the algorithm to a weak Pareto optimal point. Furthermore, under some appropriate Lipschitz continuity assumption of the gradients of objective functions, a linear convergence result for this method is also established. [ABSTRACT FROM AUTHOR]

In this paper, we propose an infeasible arc-search interior-point algorithm for solving nonlinear programming problems. Most algorithms based on interior-point methods are categorized as line search since they compute a next iterate on a straight line determined by a search direction which approximates the central path. The proposed arc-search interior-point algorithm uses an arc for the approximation. We discuss convergence properties of the proposed algorithm. We also conduct numerical experiments on the CUTEst benchmark problems and compare the performance of the proposed arc-search algorithm with that of a line-search algorithm. Numerical results indicate that the proposed arc-search algorithm reaches the optimal solution using fewer iterations but longer times than a line-search algorithm. A modification that leads to a faster arc-search algorithm is also discussed. [ABSTRACT FROM AUTHOR]

This paper is concerned with the inexact variable metric method for solving convex-constrained optimization problems. At each iteration of this method, the search direction is obtained by inexactly minimizing a strictly convex quadratic function over the closed convex feasible set. Here, we propose a new inexactness criterion for the search direction subproblems. Under mild assumptions, we prove that any accumulation point of the sequence generated by the new method is a stationary point of the problem under consideration. In order to illustrate the practical advantages of the new approach, we report some numerical experiments. In particular, we present an application where our concept of the inexact solutions is quite appealing. [ABSTRACT FROM AUTHOR]

The purpose of this paper is to present an inexact version of the scaled gradient projection method on a convex set, which is inexact in two sense. First, an inexact projection on the feasible set is computed, allowing for an appropriate relative error tolerance. Second, an inexact nonmonotone line search scheme is employed to compute a step size which defines the next iteration. It is shown that the proposed method has similar asymptotic convergence properties and iteration-complexity bounds as the usual scaled gradient projection method employing monotone line searches. [ABSTRACT FROM AUTHOR]

In this paper, we focus on the recession cone and recession function in the nonconvex case. By virtue of the recession function, we investigate the monotonicity of a function. Then, we obtain a characterization of the Lipschitz continuity for a function by using the recession function. As applications, we study the monotonicity and Lipschitz continuity properties of the optimal value function for a parametric constrained optimization problem. Moreover, we present a characterization of the nonemptiness and boundedness of the solution set for a constrained optimization problem without convexity assumption. [ABSTRACT FROM AUTHOR]

Switching-constrained optimization problems form a difficult class of mathematical programmes since their feasible set is almost disconnected while standard constraint qualifications are likely to fail at several feasible points. That is why the application of standard methods from nonlinear programming does not seem to be promising in order to solve such problems. In this paper, we adapt several relaxation methods which are well known from the numerical treatment of mathematical programmes with complementarity constraints to the setting of switching-constrained optimization. A detailed convergence analysis is provided for the adapted relaxation schemes of Scholtes as well as Kanzow and Schwartz. While Scholtes' method and the relaxation scheme of Steffensen and Ulbrich only find weakly stationary points in general, it is shown that the adapted relaxation scheme of Kanzow and Schwartz is capable of identifying Mordukhovich-stationary points of switching-constrained programmes under suitable assumptions. Some computational experiments and a numerical comparison of the proposed methods based on examples from logical programming, switching control, and portfolio optimization close the paper. [ABSTRACT FROM AUTHOR]

The paper is devoted to the existence of global optimal solutions for a general class of nonsmooth problems of constrained vector optimization without boundedness assumptions on constraint set. The main attention is paid to the two major notions of optimality in vector problems: Pareto efficiency and proper efficiency in the sense of Geoffrion. Employing adequate tools of variational analysis and generalized differentiation, we first establish relationships between the notions of properness, M-tameness, and the Palais–Smale conditions formulated for the restriction of the vector cost mapping on the constraint set. These results are instrumental to derive verifiable necessary and sufficient conditions for the existence of Pareto efficient solutions in vector optimization. Furthermore, the developed approach allows us to obtain new sufficient conditions for the existence of Geoffrion-properly efficient solutions to such constrained vector problems. [ABSTRACT FROM AUTHOR]

This paper focuses on second-order necessary optimality conditions for constrained optimization problems on Banach spaces. For problems in the classical setting, where the objective function is C 2 -smooth, we show that strengthened second-order necessary optimality conditions are valid if the constraint set is generalized polyhedral convex. For problems in a new setting, where the objective function is just assumed to be C 1 -smooth and the constraint set is generalized polyhedral convex, we establish sharp second-order necessary optimality conditions based on the Fréchet second-order subdifferential of the objective function and the second-order tangent set to the constraint set. Three examples are given to show that the used hypotheses are essential for the new theorems. Our second-order necessary optimality conditions refine and extend several existing results. [ABSTRACT FROM AUTHOR]

The paper illustrates connections between classical results of Analysis and Optimization. The focus is on new elementary proofs of Implicit Function Theorem, Lusternik's Theorem, and optimality conditions for equality constrained optimization problems. The proofs are based on Fermat's Theorem and the Weierstrass Theorem and do not use the contraction mapping principle or other advanced results of Real Analysis, so they can be used in any introductory course on Optimization or Real Analysis without the requirement of the advanced background in analysis. The paper also presents a simple proof of Implicit Function Theorem in normed linear spaces. [ABSTRACT FROM AUTHOR]

This article is concerned with the problem of minimizing a smooth function over the Stiefel manifold. In order to address this problem, we introduce two adaptive scaled gradient projection methods that incorporate scaling matrices that depend on the step-size and a parameter that controls the search direction. These iterative algorithms use a projection operator based on the QR factorization to preserve the feasibility in each iteration. However, for some particular cases, the proposals do not require the use of any projection operator. In addition, we consider a Barzilai and Borwein-like step-size combined with the Zhang–Hager nonmonotone line-search technique in order to accelerate the convergence of the proposed procedures. We proved the global convergence for these schemes, and we evaluate their effectiveness and efficiency through an extensive computational study, comparing our approaches with other state-of-the-art gradient-type algorithms. [ABSTRACT FROM AUTHOR]

We propose a convex quadratic programming (CQP) relaxation for multi-ball constrained quadratic optimization (MB). (CQP) is shown to be equivalent to semidefinite programming relaxation in the hard case. Based on (CQP), we propose an algorithm for solving (MB), which returns a solution of (MB) with an approximation bound independent of the number of constraints. The approximation algorithm is further extended to solve nonconvex quadratic optimization with more general constraints. As an application, we propose a standard quadratic programming relaxation for finding Chebyshev center of a general convex set with a guaranteed approximation bound. [ABSTRACT FROM AUTHOR]

The forward–backward algorithm is a splitting method for solving convex minimization problems of the sum of two objective functions. It has a great attention in optimization due to its broad application to many disciplines, such as image and signal processing, optimal control, regression, and classification problems. In this work, we aim to introduce new forward–backward algorithms for solving both unconstrained and constrained convex minimization problems by using linesearch technique. We discuss the convergence under mild conditions that do not depend on the Lipschitz continuity assumption of the gradient. Finally, we provide some applications to solving compressive sensing and image inpainting problems. Numerical results show that the proposed algorithm is more efficient than some algorithms in the literature. We also discuss the optimal choice of parameters in algorithms via numerical experiments. [ABSTRACT FROM AUTHOR]

In this paper, we give necessary conditions for the existence of a strict local minimum of order two for multiobjective optimization problems with equality and inequality constraints. We suppose that the objective function and the active inequality constraints are only locally Lipschitz. We consider both regular equality constraints and degenerate equality constraints. This article could be considered as a continuation of [E. Constantin, Necessary Conditions for Weak Efficiency for Nonsmooth Degenerate Multiobjective Optimization Problems, J. Global Optim, 75, 111-129, 2019]. We introduce a constraint qualification and a regularity condition, and we show that under each of them, the dual necessary conditions for a weak local minimum of the aforementioned article become of Kuhn-Tucker type. [ABSTRACT FROM AUTHOR]

A major difficulty in optimization with nonconvex constraints is to find feasible solutions. As simple examples show, the α BB-algorithm for single-objective optimization may fail to compute feasible solutions even though this algorithm is a popular method in global optimization. In this work, we introduce a filtering approach motivated by a multiobjective reformulation of the constrained optimization problem. Moreover, the multiobjective reformulation enables to identify the trade-off between constraint satisfaction and objective value which is also reflected in the quality guarantee. Numerical tests validate that we indeed can find feasible and often optimal solutions where the classical single-objective α BB method fails, i.e., it terminates without ever finding a feasible solution. [ABSTRACT FROM AUTHOR]

We propose a local convergence analysis of a primal–dual interior point algorithm for the solution of a bound-constrained optimization problem. The algorithm includes a regularization technique to prevent singularity of the matrix of the linear system at each iteration, when the second-order sufficient conditions do not hold at the solution. These conditions are replaced by a milder assumption related to a local error-bound condition. This new condition is a generalization of the one used in unconstrained optimization. We show that by an appropriate updating strategy of the barrier parameter and of the regularization parameter, the proposed algorithm owns a superlinear rate of convergence. The analysis is made thanks to a boundedness property of the inverse of the Jacobian matrix arising in interior point algorithms. An illustrative example is given to show the behavior and the gain obtained by this regularization strategy. [ABSTRACT FROM AUTHOR]

The Barzilai–Borwein (BB) step size initially proposed in the context of unconstrained optimization has become one of the most popular step choices for gradient-based methods in the optimization community. It is well-known that the powerful variational inequality can be used to characterize the first-order optimality condition of constrained optimization problems. However, a variational inequality problem is not always equivalent to a constrained optimization problem. In this paper, we follow the spirit of BB step size and propose a projection method with alternate BB step size for general variational inequalities. Although the global convergence is established under some strong conditions, a series of computational experiments on nonlinear complementarity problems, image deblurring problems and generalized Nash equilibrium problems demonstrate that the proposed almost-parameter-free projection method is more efficient than some existing state-of-the-art projection methods in the literature. [ABSTRACT FROM AUTHOR]

In this work we present a new model based on a fused Lasso approach for the multi-period portfolio selection problem in a Markowitz framework. In a multi-period setting, the investment period is partitioned into sub-periods, delimited by the rebalancing dates at which decisions are taken. The model leads to a constrained optimization problem. Two l 1 penalty terms are introduced into the objective function to reduce the costs of the investment strategy. The former is applied to portfolio weights, encouraging sparse solutions. The latter is a penalization on the difference of wealth allocated across the assets between rebalancing dates, thus it preserves the pattern of active positions with the effect of limiting the number of transactions. We solve the problem by means of the Split Bregman iteration. We show results of numerical tests performed on real data to validate our model. [ABSTRACT FROM AUTHOR]

In this paper, optimality conditions are studied for a class of constrained interval-valued optimization problems governed by path-independent curvilinear integral (mechanical work) cost functionals. Specifically, a minimal criterion of optimality for a local LU-optimal solution of the considered PDE&PDI-constrained variational control problem to be its global LU-optimal solution is formulated and proved. In addition, the main result is highlighted by an illustrative application describing the controlled behavior of an artificial neural system. [ABSTRACT FROM AUTHOR]