Search

In this paper we develop new tensor methods for unconstrained convex optimization, which solve at each iteration an auxiliary problem of minimizing convex multivariate polynomial. We analyze the simplest scheme, based on minimization of a regularized local model of the objective function, and its accelerated version obtained in the framework of estimating sequences. Their rates of convergence are compared with the worst-case lower complexity bounds for corresponding problem classes. Finally, for the third-order methods, we suggest an efficient technique for solving the auxiliary problem, which is based on the recently developed relative smoothness condition (Bauschke et al. in Math Oper Res 42:330–348, 2017; Lu et al. in SIOPT 28(1):333–354, 2018). With this elaboration, the third-order methods become implementable and very fast. The rate of convergence in terms of the function value for the accelerated third-order scheme reaches the level $$O\left( {1 \over k^4}\right) $$ O 1 k 4 , where k is the number of iterations. This is very close to the lower bound of the order $$O\left( {1 \over k^5}\right) $$ O 1 k 5 , which is also justified in this paper. At the same time, in many important cases the computational cost of one iteration of this method remains on the level typical for the second-order methods.

In order to solve the Sylvester equations more efficiently, a new four parameters positive and skew-Hermitian splitting (FPPSS) iterative method is proposed in this paper based on the previous research of the positive and skew-Hermitian splitting (PSS) iterative method. We prove that when coefficient matrix A and B satisfy certain conditions, the FPPSS iterative method is convergent in the parameter&rsquo, s value region. The numerical experiment results show that compared with previous iterative method, the FPPSS iterative method is more effective in terms of iteration number IT and runtime.

We propose a complete-search algorithm for solving a class of non-convex, possibly infinite-dimensional, optimization problems to global optimality. We assume that the optimization variables are in a bounded subset of a Hilbert space, and we determine worst-case run-time bounds for the algorithm under certain regularity conditions of the cost functional and the constraint set. Because these run-time bounds are independent of the number of optimization variables and, in particular, are valid for optimization problems with infinitely many optimization variables, we prove that the algorithm converges to an \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varepsilon $$\end{document}ε-suboptimal global solution within finite run-time for any given termination tolerance \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varepsilon > 0$$\end{document}ε>0. Finally, we illustrate these results for a problem of calculus of variations.

In a recent paper [JUKIĆ, D.: A necessary and sufficient criterion for the existence of the global minima of a continuous lower bounded function on a noncompact set, J. Comput. Appl. Math. 375 (2020)], a new existence level was introduced and then was used to obtain a necessary and sufficient criterion for the existence of the global minima of a continuous lower bounded function on a noncompact set. In this paper, we determined that existence level for the residual sum of squares of the power-law regression with an unknown location parameter, and so we obtained a necessary and sufficient condition which guarantee the existence of the least squares estimate.

International audience; Sampled-data control systems have steadily been gaining interest for their applications in automatic engineering where they are implemented as digital controllers and recently results have been obtained in optimal control theory for nonlinear sampled-data control systems and certain generalizations. In this paper we derive a Pontryagin maximum principle for general nonlinear finite-dimensional optimal sampled-data control problems with running inequality state constraints. In particular, we obtain a nonpositive averaged Hamiltonian gradient condition with the adjoint vector being a function of bounded variations. Our proof is based on the Ekeland variational principle. In general, optimal control problems with running inequality state constraints are difficult to solve using numerical methods due to the discontinuities (the jumps and the singular part) of the adjoint vector. However in our case we find that under certain general hypotheses the adjoint vector only experiences jumps at most at the sampling times and moreover the trajectory only contacts the running inequality state constraints at most at the sampling times. We call this behavior a bouncing trajectory phenomenon and it constitutes the second major focus of this paper. Finally taking advantage of the bouncing trajectory phenomenon we numerically solve three examples with different kinds of constraints and in several dimensions.

In this paper, we prove the weak convergence of a modified extragradient algorithm for solving a variational inequality problem involving a pseudomonotone operator in an infinite dimensional Hilbert space. Moreover, we establish further the R-linear rate of the convergence of the proposed algorithm with the assumption of error bound. Several numerical experiments are performed to illustrate the convergence behaviour of the new algorithm in comparisons with others. The results obtained in the paper have extended some recent results in the literature.

In this paper we revisit the problem of performing a $QZ$ step with a so-called ‘perfect shift’, which is an ‘exact’ eigenvalue of a given regular pencil $\lambda B-A$ in unreduced Hessenberg triangular form. In exact arithmetic, the $QZ$ step moves that eigenvalue to the bottom of the pencil, while the rest of the pencil is maintained in Hessenberg triangular form, which then yields a deflation of the given eigenvalue. But in finite precision the $QZ$ step gets ‘blurred’ and precludes the deflation of the given eigenvalue. In this paper we show that when we first compute the corresponding eigenvector to sufficient accuracy, then the $QZ$ step can be constructed using this eigenvector, so that the deflation is also obtained in finite precision. An important application of this technique is the computation of the index of a system of differential algebraic equations, since an exact deflation of the infinite eigenvalues is needed to impose correctly the algebraic constraints of such differential equations.

This paper deals with abstract quasi-variational-like problems, which cover vector quasi-variational problems with set-valued mappings, for instance. The main objective of this paper is to study existence results for generalized solutions of quasi-variational-like problems extending the approach by Jadamba, Khan and Sama, see (Optim. Lett. 6, 1221–1231, 2012). It is well-known that a quasi-variational-like problem can conveniently be formulated as a set-valued fixed-point problem by using the so-called variational selection. Since the values of the corresponding set-valued mapping are non-convex in general and since the existing fixed-point theorems for non-convex mappings require very stringent conditions on the data of the problem, a few existence results have been obtained in the case where the underlying constraining set is bounded or even compact. In this paper, we focus on the relationship between solutions of a quasi-variational-like problem and generalized solutions of an optimization problem minimizing the difference between inputs and outputs of the variational selection. Doing so, we can solve the latter problem by the well-known Weierstras theorem with much less restrictive assumptions imposed on the given data. This approach allows to consider problems where the variational selection is non-convex and/or the underlying constraining set is unbounded.

Recently, Bai and Zhang [Numerical Linear Algebra with Applications, 2013, 20, 425Ƀ439] constructed modulus-based synchronous multisplitting methods by an equivalent reformulation of the linear complementarity problem into a system of fixed-point equations and studied the convergence of them; Li et al. [Journal of Nanchang University (Natural Science), 2013, 37, 307Ƀ312] studied synchronous block multisplitting iteration methods; Zhang and Li [Computers and Mathematics with Application, 2014, 67, 1954Ƀ1959] analyzed and obtained the weaker convergence results for linear complementarity problems. In this paper, we generalize their algorithms and further study global relaxed modulus-based synchronous block multisplitting multi-parameters methods for linear complementarity problems. Furthermore, we give the weaker convergence results of our new method in this paper when the system matrix is a block $ H_{+}- $matrix. Therefore, new results provide a guarantee for the optimal relaxation parameters, please refer to [A. Hadjidimos, M. Lapidakis and M. Tzoumas, SIAM Journal on Matrix Analysis and Applications, 2012, 33, 97Ƀ110, (dx.doi.org/10.1137/100811222)], where optimal parameters are determined.

In this paper we study nonconvex and nonsmooth multi-block optimization over Euclidean embedded (smooth) Riemannian submanifolds with coupled linear constraints. Such optimization problems naturally arise from machine learning, statistical learning, compressive sensing, image processing, and tensor PCA, among others. By utilizing the embedding structure, we develop an ADMM-like primal-dual approach based on decoupled solvable subroutines such as linearized proximal mappings, where the duality is with respect to the embedded Euclidean spaces. First, we introduce the optimality conditions for the afore-mentioned optimization models. Then, the notion of $$\epsilon $$ -stationary solutions is introduced as a result. The main part of the paper is to show that the proposed algorithms possess an iteration complexity of $$O(1/\epsilon ^2)$$ to reach an $$\epsilon $$ -stationary solution. For prohibitively large-size tensor or machine learning models, we present a sampling-based stochastic algorithm with the same iteration complexity bound in expectation. In case the subproblems are not analytically solvable, a feasible curvilinear line-search variant of the algorithm based on retraction operators is proposed. Finally, we show specifically how the algorithms can be implemented to solve a variety of practical problems such as the NP-hard maximum bisection problem, the $$\ell _q$$ regularized sparse tensor principal component analysis and the community detection problem. Our preliminary numerical results show great potentials of the proposed methods.

The augmented Lagrangian method (ALM) is fundamental in solving convex programming problems with linear constraints. The proximal version of ALM, which regularizes ALM’s subproblem over the primal variable at each iteration by an additional positive-definite quadratic proximal term, has been well studied in the literature. In this paper we show that it is not necessary to employ a positive-definite quadratic proximal term for the proximal ALM and the convergence can be still ensured if the positive definiteness is relaxed to indefiniteness by reducing the proximal parameter. An indefinite proximal version of the ALM is thus proposed for the generic setting of convex programming problems with linear constraints. We show that our relaxation is optimal in the sense that the proximal parameter cannot be further reduced. The consideration of indefinite proximal regularization is particularly meaningful for generating larger step sizes in solving ALM’s primal subproblems. When the model under discussion is separable in the sense that its objective function consists of finitely many additive function components without coupled variables, it is desired to decompose each ALM’s subproblem over the primal variable in Jacobian manner, replacing the original one by a sequence of easier and smaller decomposed subproblems, so that parallel computation can be applied. This full Jacobian splitting version of the ALM is known to be not necessarily convergent, and it has been studied in the literature that its convergence can be ensured if all the decomposed subproblems are further regularized by sufficiently large proximal terms. But how small the proximal parameter could be is still open. The other purpose of this paper is to show the smallest proximal parameter for the full Jacobian splitting version of ALM for solving multi-block separable convex minimization models.

In this paper, we present a smoothing Newton method for solving the monotone weighted complementarity problem (WCP). In each iteration of our method, the iterative direction is achieved by solving a system of linear equations and the iterative step length is achieved by adopting a line search. A feature of the line search criteria used in this paper is that monotone and nonmonotone line search are mixed used. The proposed method is new even when the WCP reduces to the standard complementarity problem. Particularly, the proposed method is proved to possess the global convergence under a weak assumption. The preliminary experimental results show the effectiveness and robustness of the proposed method for solving the concerned WCP.

Variants of the coordinate descent approach for minimizing a nonlinear function are distinguished in part by the order in which coordinates are considered for relaxation. Three common orderings are cyclic (CCD), in which we cycle through the components of $x$ in order; randomized (RCD), in which the component to update is selected randomly and independently at each iteration; and random-permutations cyclic (RPCD), which differs from CCD only in that a random permutation is applied to the variables at the start of each cycle. Known convergence guarantees are weaker for CCD and RPCD than for RCD, though in most practical cases, computational performance is similar among all these variants. There is a certain type of quadratic function for which CCD is significantly slower than for RCD; a recent paper by Sun & Ye (2016, Worst-case complexity of cyclic coordinate descent: $O(n^2)$ gap with randomized version. Technical Report. Stanford, CA: Department of Management Science and Engineering, Stanford University. arXiv:1604.07130) has explored the poor behavior of CCD on functions of this type. The RPCD approach performs well on these functions, even better than RCD in a certain regime. This paper explains the good behavior of RPCD with a tight analysis.

The purpose of this paper is to give some new Karush–Kuhn–Tucker-type necessary optimality conditions for nonsmooth semi-infinite problems. Moreover, we present some suitable examples for our results. The paper is organized by Frechet and Mordukhovich subdifferentials.

Stochastic variational inequalities (SVI) provide a unified framework for the study of a general class of nonlinear optimization and Nash-type equilibrium problems with uncertain model data. Often the true solution to an SVI cannot be found directly and must be approximated. This paper considers the use of a sample average approximation (SAA), and proposes a new method to compute confidence intervals for individual components of the true SVI solution based on the asymptotic distribution of SAA solutions. We estimate the asymptotic distribution based on one SAA solution instead of generating multiple SAA solutions, and can handle inequality constraints without requiring the strict complementarity condition in the standard nonlinear programming setting. The method in this paper uses the confidence regions to guide the selection of a single piece of a piecewise linear function that governs the asymptotic distribution of SAA solutions, and does not rely on convergence rates of the SAA solutions in probability. It also provides options to control the computation procedure and investigate effects of certain key estimates on the intervals.

In this paper, we establish the convergence of the proximal alternating direction method of multipliers (ADMM) and block coordinate descent (BCD) method for nonseparable minimization models with quadratic coupling terms. The novel convergence results presented in this paper answer several open questions that have been the subject of considerable discussion. We firstly extend the 2-block proximal ADMM to linearly constrained convex optimization with a coupled quadratic objective function, an area where theoretical understanding is currently lacking, and prove that the sequence generated by the proximal ADMM converges in point-wise manner to a primal-dual solution pair. Moreover, we apply randomly permuted ADMM (RPADMM) to nonseparable multi-block convex optimization, and prove its expected convergence for a class of nonseparable quadratic programming problems. When the linear constraint vanishes, the 2-block proximal ADMM and RPADMM reduce to the 2-block cyclic proximal BCD method and randomly permuted BCD (RPBCD). Our study provides the first iterate convergence result for 2-block cyclic proximal BCD without assuming the boundedness of the iterates. We also theoretically establish the expected iterate convergence result concerning multi-block RPBCD for convex quadratic optimization. In addition, we demonstrate that RPBCD may have a worse convergence rate than cyclic proximal BCD for 2-block convex quadratic minimization problems. Although the results on RPADMM and RPBCD are restricted to quadratic minimization models, they provide some interesting insights: (1) random permutation makes ADMM and BCD more robust for multi-block convex minimization problems; (2) cyclic BCD may outperform RPBCD for “nice” problems, and RPBCD should be applied with caution when solving general convex optimization problems especially with a few blocks.

This paper gives simple proofs of Sylvester (` = 2) and Frobenius (` = 3) inequalities. Moreover, a new sufficient condition for the equality of the Frobenius inequality is provided. In addition, an extension for ` > 3 matrices and an application to generalized inverses are provided., This paper has been partially supported by Ministerio de Economia y Competitividad (Grant DGI MTM2013-43678P and Red de Excelencia MTM2015-68805-REDT). The author thanks the referees for their valuable suggestions.

In this paper, we introduce a system of split variational inequality problems in real Hilbert spaces. Using a projection method, we propose an iterative algorithm for solving this system of split variational inequality problems. Further, we prove that the sequence generated by the iterative algorithm converges strongly to a solution of the system of split variational inequality problems. Furthermore, we discuss some consequences of the main result. The iterative algorithms and results presented in this paper generalize, unify and improve the previously known results of this area.

Adaptive regularization with cubics (ARC) is an algorithm for unconstrained, non-convex optimization. Akin to the popular trust-region method, its iterations can be thought of as approximate, safe-guarded Newton steps. For cost functions with Lipschitz continuous Hessian, ARC has optimal iteration complexity, in the sense that it produces an iterate with gradient smaller than $\varepsilon$ in $O(1/\varepsilon^{1.5})$ iterations. For the same price, it can also guarantee a Hessian with smallest eigenvalue larger than $-\varepsilon^{1/2}$. In this paper, we study a generalization of ARC to optimization on Riemannian manifolds. In particular, we generalize the iteration complexity results to this richer framework. Our central contribution lies in the identification of appropriate manifold-specific assumptions that allow us to secure these complexity guarantees both when using the exponential map and when using a general retraction. A substantial part of the paper is devoted to studying these assumptions---relevant beyond ARC---and providing user-friendly sufficient conditions for them. Numerical experiments are encouraging., Comment: 48 pages, 3 figures

This paper is devoted to the design of an efficient and convergent {semi-proximal} alternating direction method of multipliers (ADMM) for finding a solution of low to medium accuracy to convex quadratic conic programming and related problems. For this class of problems, the convergent two block semi-proximal ADMM can be employed to solve their primal form in a straightforward way. However, it is known that it is more efficient to apply the directly extended multi-block semi-proximal ADMM, though its convergence is not guaranteed, to the dual form of these problems. Naturally, one may ask the following question: can one construct a convergent multi-block semi-proximal ADMM that is more efficient than the directly extended semi-proximal ADMM? Indeed, for linear conic programming with 4-block constraints this has been shown to be achievable in a recent paper by Sun, Toh and Yang [arXiv preprint arXiv:1404.5378, (2014)]. Inspired by the aforementioned work and with the convex quadratic conic programming in mind, we propose a Schur complement based convergent semi-proximal ADMM for solving convex programming problems, with a coupling linear equality constraint, whose objective function is the sum of two proper closed convex functions plus an arbitrary number of convex quadratic or linear functions. Our convergent semi-proximal ADMM is particularly suitable for solving convex quadratic semidefinite programming (QSDP) with constraints consisting of linear equalities, a positive semidefinite cone and a simple convex polyhedral set. The efficiency of our proposed algorithm is demonstrated by numerical experiments on various examples including QSDP., 39 pages, 1 figure, 4 tables

Non-convex optimization with local search heuristics has been widely used in machine learning, achieving many state-of-art results. It becomes increasingly important to understand why they can work for these NP-hard problems on typical data. The landscape of many objective functions in learning has been conjectured to have the geometric property that "all local optima are (approximately) global optima", and thus they can be solved efficiently by local search algorithms. However, establishing such property can be very difficult. In this paper, we analyze the optimization landscape of the random over-complete tensor decomposition problem, which has many applications in unsupervised learning, especially in learning latent variable models. In practice, it can be efficiently solved by gradient ascent on a non-convex objective. We show that for any small constant $\epsilon > 0$, among the set of points with function values $(1+\epsilon)$-factor larger than the expectation of the function, all the local maxima are approximate global maxima. Previously, the best-known result only characterizes the geometry in small neighborhoods around the true components. Our result implies that even with an initialization that is barely better than the random guess, the gradient ascent algorithm is guaranteed to solve this problem. Our main technique uses Kac-Rice formula and random matrix theory. To our best knowledge, this is the first time when Kac-Rice formula is successfully applied to counting the number of local minima of a highly-structured random polynomial with dependent coefficients., Comment: Best paper in the NIPS 2016 Workshop on Nonconvex Optimization for Machine Learning: Theory and Practice. In submission

This paper deals with the nonmonotone projection algorithm for constrained nonlinear equations. For some starting points, the previous projection algorithms for the problem may encounter slow convergence which is related to the monotone behavior of the iterative sequence as well as the iterative direction. To circumvent this situation, we adopt the nonmonotone technique introduced by Dang to develop a nonmonotone projection algorithm. After constructing the nonmonotone projection algorithm, we show its convergence under some suitable condition. Preliminary numerical experiment is reported at the end of this paper, from which we can see that the algorithm we propose converges more quickly than that of the usual projection algorithm for some starting points.

The Colebrook-White equation is often used for calculation of the friction factor in turbulent regimes; it has succeeded in attracting a great deal of attention from researchers. The Colebrook–White equation is a complex equation where the computation of the friction factor is not direct, and there is a need for trial-error methods or graphical solutions; on the other hand, several researchers have attempted to alter the Colebrook-White equation by explicit formulas with the hope of achieving zero-percent (0%) maximum deviation, among them Dejan Brkić and Pavel Praks. The goal of this paper is to discuss the results proposed by the authors in their paper:” Accurate and Efficient Explicit Approximations of the Colebrook Flow Friction Equation Based on the Wright ω-Function” and to propose more accurate formulas.

In a paper Cheung, Cucker and Peña (in press) [5] that can be seen as the first part of this one, we extended the well-known condition numbers for polyhedral conic systems C(A) Renegar (1994, 1995) [7–9] and C(A) Cheung and Cucker (2001) [3] to versions C¯(A) and C¯(A) that are finite for all input matrices A∈Rn×m. In this paper we compare C¯(A) and C¯(A) with other condition measures for the same problem that are also always finite.

In this paper, we propose and analyze a trust-region model-based algorithm for solving unconstrained stochastic optimization problems. Our framework utilizes random models of an objective function $f(x)$, obtained from stochastic observations of the function or its gradient. Our method also utilizes estimates of function values to gauge progress that is being made. The convergence analysis relies on requirements that these models and these estimates are sufficiently accurate with sufficiently high, but fixed, probability. Beyond these conditions, no assumptions are made on how these models and estimates are generated. Under these general conditions we show an almost sure global convergence of the method to a first order stationary point. In the second part of the paper, we present examples of generating sufficiently accurate random models under biased or unbiased noise assumptions. Lastly, we present some computational results showing the benefits of the proposed method compared to existing approaches that are based on sample averaging or stochastic gradients., Comment: Revised version posted September 23, 2016. Originally posted April 17, 2015

In this paper, we present a new framework of bi-level unconstrained minimization for development of accelerated methods in Convex Programming. These methods use approximations of the high-order proximal points, which are solutions of some auxiliary parametric optimization problems. For computing these points, we can use different methods, and, in particular, the lower-order schemes. This opens a possibility for the latter methods to overpass traditional limits of the Complexity Theory. As an example, we obtain a new second-order method with the convergence rate $$O\left( k^{-4}\right) $$ O k - 4 , where k is the iteration counter. This rate is better than the maximal possible rate of convergence for this type of methods, as applied to functions with Lipschitz continuous Hessian. We also present new methods with the exact auxiliary search procedure, which have the rate of convergence $$O\left( k^{-(3p+1)/ 2}\right) $$ O k - ( 3 p + 1 ) / 2 , where $$p \ge 1$$ p ≥ 1 is the order of the proximal operator. The auxiliary problem at each iteration of these schemes is convex.

We study approximation of multivariate functions defined over Rd. We assume that all rth order partial derivatives of the functions considered are continuous and uniformly bounded. Approximation algorithms U(f) only use the values of f or its partial derivatives up to order r. We want to recover the function f with small error measured in a weighted Lq norm with a weight function ρ. We study the worst case (information) complexity which is equal to the minimal number of function and derivative evaluations needed to obtain error ε. We provide necessary and sufficient conditions in terms of the weight ρ and the parameters q and r for the weighted approximation problem to have finite complexity. We also provide conditions guaranteeing that the complexity is of the same order as the complexity of the classical approximation problem over a finite domain. Since the complexity of the weighted integration problem is equivalent to the complexity of the weighted approximation problem with q=1, the results of this paper also hold for weighted integration. This paper is a continuation of [7], where weighted approximation over R was studied.

The conjugate gradient (CG) method is a method to solve unconstrained optimization problems. Moreover CG method can be applied in medical science, industry, neural network, and many others. In this paper a new three term CG method is proposed. The new CG formula is constructed based on DL and WYL CG formulas to be non-negative and inherits the properties of HS formula. The new modification satisfies the convergence properties and the sufficient descent property. The numerical results show that the new modification is more efficient than DL, WYL, and CG-Descent formulas. We use more than 200 functions from CUTEst library to compare the results between these methods in term of number of iterations, function evaluations, gradient evaluations, and CPU time.

In this paper, we inspect the global asymptotic stability (GAS) of Ebola–epidemic using a modified Susceptible-Infected-Deceased-Pathogen (SIDP) model with perturbation. The feasible region is obtained, and we determine the basic reproduction number using the next-generation matrix method. The GAS of the disease-free and endemic equilibria are discuss and provide parameter conditions for which the stability exists. We show theoretically that the contaminated environmental Ebola human–deceased transmission model is GAS with and without probability. Numerical simulation to support our theoretical findings are presented.

Many iterative methods in applied mathematics can be thought of as fixed-point iterations, and such algorithms are usually analyzed analytically, with inequalities. In this paper, we present a geometric approach to analyzing contractive and nonexpansive fixed point iterations with a new tool called the scaled relative graph (SRG). The SRG provides a correspondence between nonlinear operators and subsets of the 2D plane. Under this framework, a geometric argument in the 2D plane becomes a rigorous proof of convergence., Published in Mathematical Programming

In this paper, a novel stochastic extra-step quasi-Newton method is developed to solve a class of nonsmooth nonconvex composite optimization problems. We assume that the gradient of the smooth part of the objective function can only be approximated by stochastic oracles. The proposed method combines general stochastic higher order steps derived from an underlying proximal type fixed-point equation with additional stochastic proximal gradient steps to guarantee convergence. Based on suitable bounds on the step sizes, we establish global convergence to stationary points in expectation and an extension of the approach using variance reduction techniques is discussed. Motivated by large-scale and big data applications, we investigate a stochastic coordinate-type quasi-Newton scheme that allows to generate cheap and tractable stochastic higher order directions. Finally, numerical results on large-scale logistic regression and deep learning problems show that our proposed algorithm compares favorably with other state-of-the-art methods.

According to the length-preserving projection and Euler discretization method, biorthogonal wavelet function on a smooth curve C is constructed in this paper, such as a logarithm curve. The properties of biorthogonal wavelet filters on a smooth curve C are discussed, such as induced refinable equation and symmetry. Moreover, an example is given for discussing the biorthogonal scaling function and its dual on a logarithm curve C . Finally, a numerical application is given for dealing with financial data.

Generalized Sylvester quaternion matrix equation with some constrictions is explored in this paper. A novel expression of the general solution of the constraint system is given with some necessary and sufficient conditions. Its Cramer’s rule is derived within the framework of the theory of row-column quaternion determinants. An example is given to justify obtained theoretical results.

This paper focuses on the problem of convex constraint nonlinear equations involving monotone operators in Euclidean space. A Fletcher and Reeves type derivative-free conjugate gradient method is proposed. The proposed method is designed to ensure the descent property of the search direction at each iteration. Furthermore, the convergence of the proposed method is proved under the assumption that the underlying operator is monotone and Lipschitz continuous. The numerical results show that the method is efficient for the given test problems.

As it is well known the problem of solving the Fredholm integral equation of the first kind belongs to the class of ill-posed problems. The Tikhonov regularization method is well known. This method is usually applied to an integral equation and a system of linear algebraic equations. The authors firstly propose to reduce the integral equation of the first kind to a system of linear algebraic equations. This system is usually extremely ill-posed. Therefore, it is necessary to carry out the Tikhonov regularization for the system of equations. In this paper, to form a system of linear algebraic equations, local polynomial and non-polynomial spline approximations of the second order of approximation are used. The results of numerical experiments are presented.

Quadratically constrained quadratic programs (QCQPs) are a fundamental class of optimization problems well-known to be NP-hard in general. In this paper we study conditions under which the standard semidefinite program (SDP) relaxation of a QCQP is tight. We begin by outlining a general framework for proving such sufficient conditions. Then using this framework, we show that the SDP relaxation is tight whenever the quadratic eigenvalue multiplicity, a parameter capturing the amount of symmetry present in a given problem, is large enough. We present similar sufficient conditions under which the projected epigraph of the SDP gives the convex hull of the epigraph in the original QCQP. Our results also imply new sufficient conditions for the tightness (as well as convex hull exactness) of a second order cone program relaxation of simultaneously diagonalizable QCQPs.

The purpose of the paper is to find an approximate solution of the two-dimensional nonlinear fuzzy Volterra integral equation, as homotopy analysis method (HAM) is applied. Studied equation is converted to a nonlinear system of Volterra integral equations in a crisp case. Using HAM we find approximate solution of this system and hence obtain an approximation for the fuzzy solution of the nonlinear fuzzy Volterra integral equation. The convergence of the proposed method is proved. An error estimate between the exact and the approximate solution is found. The validity and applicability of the HAM are illustrated by a numerical example.

In this paper, we propose a simple method to generate generalized Gaussian noises using the inverse transform of cumulative distribution. This inverse is expressible by means of the inverse incomplete Gamma function. Since the implementation of Newton's method is rather simple, for approximating inverse incomplete Gamma function, we propose a better and new initial value exploiting the close relationship between the incomplete Gamma function and its piecewise linear interpolant. The numerical results highlight that the proposed method simulates well the univariate and bivariate generalized Gaussian noises.

In this paper, we introduce a new algorithm for solving pseudomonotone variational inequalities with a Lipschitz-type condition in a real Hilbert space. The algorithm is constructed around two algorithms: the subgradient extragradient algorithm and the inertial algorithm. The proposed algorithm uses a new step size rule based on local operator information rather than its Lipschitz constant or any other line search scheme and functions without any knowledge of the Lipschitz constant of an operator. The strong convergence of the algorithm is provided. To determine the computational performance of our algorithm, some numerical results are presented.

This paper has been withdrawn by the author due to its publication, This paper has been withdrawn

This paper considers the nonlinear symmetric conic programming (NSCP) problems. Firstly, a type of strong sufficient optimality condition for NSCP problems in terms of a linear-quadratic term is introduced. Then, a sufficient condition of the nonsingularity of Clarke’s generalized Jacobian of the Karush–Kuhn–Tucker (KKT) system is demonstrated. At last, as an application, this property is used to obtain the local convergence properties of a sequential quadratic programming- (SQP-) type method.

In this paper, we introduce some new notions of quasi efficiency and quasi proper efficiency for multiobjective optimization problems that reduce to the most important concepts of approximate and quasi efficient solutions given up to now. We establish main properties and provide characterizations for these solutions by linear and nonlinear scalarizations. With the help of quasi efficient solutions, a generalized subdifferential of a vector mapping is introduced, which generates a number of approximate subdifferentials frequently used in optimization in a unifying way. The generalized subdifferential is related to the classical subdifferential of real functions by the method of scalarization. An application of generalized subdifferential to express optimality conditions for quasi efficient solutions is also given.

The application of the local polynomial and non-polynomial to the construction of methods for numerically solving the heat conduction problem is discussed. The non-polynomial splines are used here to approximate the partial derivatives. Formulas for numerical differentiation based on the application of the nonpolynomial splines of the fourth order of approximation are constructed. Particular attention is paid to polynomial, trigonometric, exponential, polynomial-trigonometric and polynomial-exponential splines. This approach allows us to construct explicit and implicit difference schemes. The main focus of the paper is on implicit difference scheme. New approximations with splines of the Lagrange and Hermite type with new properties are obtained. These approximations take into account the first and second derivatives of the function being approximated. Numerical examples are given.

This work is concerned with the classical problem of finding a zero of a sum of maximal monotone operators. For the projective splitting framework recently proposed by Combettes and Eckstein, we show how to replace the fundamental subproblem calculation using a backward step with one based on two forward steps. The resulting algorithms have the same kind of coordination procedure and can be implemented in the same block-iterative and highly flexible manner, but may perform backward steps on some operators and forward steps on others. Prior algorithms in the projective splitting family have used only backward steps. Forward steps can be used for any Lipschitz-continuous operators provided the stepsize is bounded by the inverse of the Lipschitz constant. If the Lipschitz constant is unknown, a simple backtracking linesearch procedure may be used. For affine operators, the stepsize can be chosen adaptively without knowledge of the Lipschitz constant and without any additional forward steps. We close the paper by empirically studying the performance of several kinds of splitting algorithms on a large-scale rare feature selection problem.

This paper introduces a new hybrid fractional model for image denoising. This proposed model is a combination of two models Rudin–Osher–Fatemi and fractional-order total variation. We try to use the advantages of two mentioned models. In this regard, after introducing an appropriate norm space, we prove the existence and uniqueness of the presented model. Furthermore, finite difference method is employed for numerically solving the obtained equation. Finally, the results illustrate the efficiency of the proposed model that yields good visual effects and a better signal-to-noise ratio.

A Grassmannian fusion frame is an optimal configuration of subspaces of a given vector space, that are useful in some applications related to representing data in signal processing. Grassmannian fusion frames are robust against noise and erasures when the signal is reconstructed. In this paper, we present an approach to construct optimal Grassmannian fusion frames based on a given Grassmannian frame. We also analyse an algorithm for sparse fusion frames which was introduced by Calderbank et al. and present necessary and sufficient conditions for the output of that algorithm to be an optimal Grassmannian fusion frame.

A linear mapping ϕ from an algebra $${\cal A}$$ into its bimodule $${\cal M}$$ is called a centralizable mapping at G ∈ $${\cal A}$$ if ϕ(AB) = ϕ(A)B = Aϕ(B) for each A and B in $${\cal A}$$ with AB = G. In this paper, we prove that if $${\cal M}$$ is a von Neumann algebra without direct summands of type I1 and type II, $${\cal A}$$ is a *-subalgebra with $${\cal M}$$ ⊆ $${\cal A}$$ ⊆ LS ( $${\cal M}$$ ) and G is a fixed element in $${\cal A}$$ , then every continuous (with respect to the local measure topology t( $${\cal M}$$ )) centralizable mapping at G from $${\cal A}$$ into $${\cal M}$$ is a centralizer.

The BFGS method is one of the most efficient quasi-Newton methods for solving small- and medium-size unconstrained optimization problems. For the sake of exploring its more interesting properties, a modified two-parameter scaled BFGS method is stated in this paper. The intention of the modified scaled BFGS method is to improve the eigenvalues structure of the BFGS update. In this method, the first two terms and the last term of the standard BFGS update formula are scaled with two different positive parameters, and the new value of yk is given. Meanwhile, Yuan-Wei-Lu line search is also proposed. Under the mentioned line search, the modified two-parameter scaled BFGS method is globally convergent for nonconvex functions. The extensive numerical experiments show that this form of the scaled BFGS method outperforms the standard BFGS method or some similar scaled methods.

This paper focuses on the problem of variational inequalities with monotone operators in real Hilbert space. The Tseng algorithm constructed by Thong replaced a high-precision step. Thus, a new Tseng-like gradient method is constructed, and the convergence of the algorithm is proved, and the convergence performance is higher.

This paper develops new extremal principles of variational analysis that are motivated by applications to constrained problems of stochastic programming and semi-infinite programming without smoothness and/or convexity assumptions. These extremal principles concern measurable set-valued mappings/multifunctions with values in finite-dimensional spaces and are established in both approximate and exact forms. The obtained principles are instrumental to derive via variational approaches integral representations and upper estimates of regular and limiting normals cones to essential intersections of sets defined by measurable multifunctions, which are in turn crucial for novel applications to stochastic and semi-infinite programming., Comment: 26 pages