Search

For high dimensional sparse linear regression problems, we propose a sequential convex relaxation algorithm (iSCRA-TL1) by solving inexactly a sequence of truncated $\ell_1$-norm regularized minimization problems, in which the working index sets are constructed iteratively with an adaptive strategy. We employ the robust restricted null space property and sequential restricted null space property (rRNSP and rSRNSP) to provide the theoretical certificates of iSCRA-TL1. Specifically, under a mild rRNSP or rSRNSP, iSCRA-TL1 is shown to identify the support of the true $r$-sparse vector by solving at most $r$ truncated $\ell_1$-norm regularized problems, and the $\ell_1$-norm error bound of its iterates from the oracle solution is also established. As a consequence, an oracle estimator of high-dimensional linear regression problems can be achieved by solving at most $r\!+\!1$ truncated $\ell_1$-norm regularized problems. To the best of our knowledge, this is the first sequential convex relaxation algorithm to produce an oracle estimator under a weaker NSP condition within a specific number of steps, provided that the Lasso estimator lacks high quality, say, the supports of its first $r$ largest (in modulus) entries do not coincide with those of the true vector., Comment: 38 pages, 4 figures

This paper concerns the tilt stability for the minimization of the sum of a twice continuously differentiable matrix-valued function and the Ky-Fan $\kappa$-norm. By using the expression of second subderivative of the Ky-Fan $\kappa$-norm, we derive a verifiable criterion to identify the tilt stability of a local minimum for this class of nonconvex and nonsmooth problems. As a byproduct, a practical criterion is achieved for the tilt stable solution of the nuclear-norm regularized minimization., Comment: 30pages

This note concerns a class of nonmonotone descent methods for minimizing a proper lower semicontinuous Kurdyka-{\L}$\ddot{o}$jasiewicz (KL) function $\Phi$, whose iterate sequence obeys the ZH-type nonmonotone decrease condition and a relative error condition. We prove that the iterate sequence converges to a critical point of $\Phi$, and if $\Phi$ has the KL property of exponent $\theta\in(0,1)$ at this critical point, the convergence has a linear rate for $\theta\in(0,1/2]$ and a sublinear rate of exponent $\frac{1-\theta}{1-2\theta}$ for $\theta\in(1/2,1)$. Our results first resolve the full convergence of the iterate sequence generated by the ZH-type nonmonotone descent method for nonconvex and nonsmooth optimization problems, and extend the full convergence of monotone descent methods for KL optimization problems to the ZH-type nonmonotone descent method.

This paper concerns a class of low-rank composite factorization models arising from matrix completion. For this nonconvex and nonsmooth optimization problem, we propose a proximal alternating minimization algorithm (PAMA) with subspace correction, in which a subspace correction step is imposed on every proximal subproblem so as to guarantee that the corrected proximal subproblem has a closed-form solution. For this subspace correction PAMA, we prove the subsequence convergence of the iterate sequence, and establish the convergence of the whole iterate sequence and the column subspace sequences of factor pairs under the KL property of objective function and a restrictive condition that holds automatically for the column $\ell_{2,0}$-norm function. Numerical comparison with the proximal alternating linearized minimization method on one-bit matrix completion problems indicates that PAMA has an advantage in seeking lower relative error within less time., Comment: 28 pages, 3 figures

This paper concerns a class of composite image reconstruction models for impluse noise removal, which is rather general and covers existing convex and nonconvex models proposed for reconstructing images with impluse noise. For this nonconvex and nonsmooth optimization problem, we propose a proximal majorization-minimization (MM) algorithm with an implementable inexactness criterion by seeking in each step an inexact minimizer of a strongly convex majorization of the objective function, and establish the convergence of the iterate sequence under the KL assumption on the constructed potential function. This inexact proximal MM method is applied to handle gray image deblurring and color image inpainting problems, for which the associated potential function satisfy the required KL assumption. Numerical comparisons with two state-of-art solvers for image deblurring and inpainting tasks validate the efficiency of the proposed algorithm and models.

This paper concerns the quadratic assignment problem (QAP), a class of challenging combinatorial optimization problems. We provide an equivalent rank-one doubly nonnegative (DNN) reformulation with fewer equality constraints, and derive the local error bounds for its feasible set. By leveraging these error bounds, we prove that the penalty problem induced by the difference of convexity (DC) reformulation of the rank-one constraint is a global exact penalty, and so is the penalty problem for its Burer-Monteiro (BM) factorization. As a byproduct, we verify that the penalty problem for the rank-one DNN reformulation proposed in \cite{Jiang21} is a global exact penalty without the calmness assumption. Then, we develop a continuous relaxation approach by seeking approximate stationary points of a finite number of penalty problems for the BM factorization with an augmented Lagrangian method, whose asymptotic convergence certificate is also provided under a mild condition. Numerical comparison with Gurobi for \textbf{131} benchmark instances validates the efficiency of the proposed DC exact penalty approach.

We are concerned with structured $\ell_0$-norms regularization problems, with a twice continuously differentiable loss function and a box constraint. This class of problems have a wide range of applications in statistics, machine learning and image processing. To the best of our knowledge, there is no effective algorithm in the literature for solving them. In this paper, we first obtain a polynomial-time algorithm to find a point in the proximal mapping of the fused $\ell_0$-norms with a box constraint based on dynamic programming principle. We then propose a hybrid algorithm of proximal gradient method and inexact projected regularized Newton method to solve structured $\ell_0$-norms regularization problems. The whole sequence generated by the algorithm is shown to be convergent by virtue of a non-degeneracy condition, a curvature condition and a Kurdyka-{\L}ojasiewicz property. A superlinear convergence rate of the iterates is established under a locally H\"{o}lderian error bound condition on a second-order stationary point set, without requiring the local optimality of the limit point. Finally, numerical experiments are conducted to highlight the features of our considered model, and the superiority of our proposed algorithm.

This paper concerns the composite problem of minimizing the sum of a twice continuously differentiable function $f$ and a nonsmooth convex function. For this class of nonconvex and nonsmooth problems, by leveraging a practical inexactness criterion and a novel selection strategy for iterates, we propose an inexact $q$-order regularized proximal Newton method for $q\in[2,3]$, which becomes an inexact cubic regularization (CR) method for $q=3$. We prove that the whole iterate sequence converges to a stationary point for the KL objective function; and when the objective function has the KL property of exponent $\theta\in(0,\frac{q-1}{q})$, the convergence has a local $Q$-superlinear rate of order $\frac{q-1}{\theta q}$. In particular, under a local H\"{o}lderian error bound of order $\gamma\in(\frac{1}{q-1},1]$ on a second-order stationary point set, we show that the iterate and objective value sequences converge to a second-order stationary point and a second-order stationary value, respectively, with a local $Q$-superlinear rate of order $\gamma(q\!-\!1)$, specified as the $Q$-quadratic rate for $q=3$ and $\gamma=1$. This is the first practical inexact CR method with $Q$-quadratic convergence rate for nonconvex composite optimization. We validate the efficiency of the CR method with ZeroFPR as the inner solver by applying it to composite optimization problems with highly nonlinear $f$.

Either RGB images or inertial signals have been used for the task of motion capture (mocap), but combining them together is a new and interesting topic. We believe that the combination is complementary and able to solve the inherent difficulties of using one modality input, including occlusions, extreme lighting/texture, and out-of-view for visual mocap and global drifts for inertial mocap. To this end, we propose a method that fuses monocular images and sparse IMUs for real-time human motion capture. Our method contains a dual coordinate strategy to fully explore the IMU signals with different goals in motion capture. To be specific, besides one branch transforming the IMU signals to the camera coordinate system to combine with the image information, there is another branch to learn from the IMU signals in the body root coordinate system to better estimate body poses. Furthermore, a hidden state feedback mechanism is proposed for both two branches to compensate for their own drawbacks in extreme input cases. Thus our method can easily switch between the two kinds of signals or combine them in different cases to achieve a robust mocap. %The two divided parts can help each other for better mocap results under different conditions. Quantitative and qualitative results demonstrate that by delicately designing the fusion method, our technique significantly outperforms the state-of-the-art vision, IMU, and combined methods on both global orientation and local pose estimation. Our codes are available for research at https://shaohua-pan.github.io/robustcap-page/., Comment: Accepted by SIGGRAPH ASIA 2023. Project page: https://shaohua-pan.github.io/robustcap-page/

This paper concerns the minimization of the sum of a twice continuously differentiable function $f$ and a nonsmooth convex function $g$ without closed-form proximal mapping. For this class of nonconvex and nonsmooth problems, we propose a line-search based variable metric inexact proximal gradient (VMiPG) method with uniformly bounded positive definite variable metric linear operators. This method computes in each step an inexact minimizer of a strongly convex model such that the difference between its objective value and the optimal value is controlled by its squared distance from the current iterate, and then seeks an appropriate step-size along the obtained direction with an armijo line-search criterion. We prove that the iterate sequence converges to a stationary point when $f$ and $g$ are definable in the same o-minimal structure over the real field $(\mathbb{R},+,\cdot)$, and if addition the objective function $f+g$ is a KL function of exponent $1/2$, the convergence has a local R-linear rate. The proposed VMiPG method with the variable metric linear operator constructed by the Hessian of the function $f$ is applied to the scenario that $f$ and $g$ have common composite structure, and numerical comparison with a state-of-art variable metric line-search algorithm indicates that the Hessian-based VMiPG method has a remarkable advantage in terms of the quality of objective values and the running time for those difficult problems such as high-dimensional fused weighted-lasso regressions.

This paper focuses on a class of binary orthogonal optimization problems frequently arising in semantic hashing. Consider that this class of problems may have an empty feasible set, rendering them not well-defined. We introduce an equivalent model involving a restricted Stiefel manifold and a matrix box set, and then investigate its penalty problems induced by the $\ell_1$-distance from the box set and its Moreau envelope. The two penalty problems are always well-defined. Moreover, they serve as the global exact penalties provided that the original feasible set is non-empty. Notably, the penalty problem induced by the Moreau envelope is a smooth optimization over an embedded submanifold with a favorable structure. We develop a retraction-based line-search Riemannian gradient method to address the penalty problem. Finally, the proposed method is applied to supervised and unsupervised hashing tasks and is compared with several popular methods on the MNIST and CIFAR-10 datasets. The numerical comparisons reveal that our algorithm is significantly superior to other solvers in terms of feasibility violation, and it is comparable even superior to others in terms of evaluation metrics related to the Hamming distance.

Human and environment sensing are two important topics in Computer Vision and Graphics. Human motion is often captured by inertial sensors, while the environment is mostly reconstructed using cameras. We integrate the two techniques together in EgoLocate, a system that simultaneously performs human motion capture (mocap), localization, and mapping in real time from sparse body-mounted sensors, including 6 inertial measurement units (IMUs) and a monocular phone camera. On one hand, inertial mocap suffers from large translation drift due to the lack of the global positioning signal. EgoLocate leverages image-based simultaneous localization and mapping (SLAM) techniques to locate the human in the reconstructed scene. On the other hand, SLAM often fails when the visual feature is poor. EgoLocate involves inertial mocap to provide a strong prior for the camera motion. Experiments show that localization, a key challenge for both two fields, is largely improved by our technique, compared with the state of the art of the two fields. Our codes are available for research at https://xinyu-yi.github.io/EgoLocate/., Comment: Accepted by SIGGRAPH 2023. Project page: https://xinyu-yi.github.io/EgoLocate/

This paper concerns a class of DC composite optimization problems which, as an extension of convex composite optimization problems and DC programs with nonsmooth components, often arises in robust factorization models of low-rank matrix recovery. For this class of nonconvex and nonsmooth problems, we propose an inexact linearized proximal algorithm (iLPA) by computing in each step an inexact minimizer of a strongly convex majorization constructed with a partial linearization of their objective functions at the current iterate, and establish the convergence of the generated iterate sequence under the Kurdyka-\L\"ojasiewicz (KL) property of a potential function. In particular, by leveraging the composite structure, we provide a verifiable condition for the potential function to have the KL property of exponent $1/2$ at the limit point, so for the iterate sequence to have a local R-linear convergence rate. Finally, we apply the proposed iLPA to a robust factorization model for matrix completions with outliers and non-uniform sampling, and numerical comparison with a proximal alternating minimization (PAM) method confirms iLPA yields the comparable relative errors or NMAEs within less running time, especially for large-scale real data.

One-bit compressed sensing is very popular in signal processing and communications due to its low storage costs and low hardware complexity, but it is a challenging task to recover the signal by using the one-bit information. In this paper, we propose a zero-norm regularized smooth difference of convexity (DC) loss model and derive a family of equivalent nonconvex surrogates covering the MCP and SCAD surrogates as special cases. Compared to the existing models, the new model and its SCAD surrogate have better robustness. To compute their $\tau$-stationary points, we develop a proximal gradient algorithm with extrapolation and establish the convergence of the whole iterate sequence. Also, the convergence is proved to have a linear rate under a mild condition by studying the KL property of exponent 0 of the models. Numerical comparisons with several state-of-art methods show that in terms of the quality of solution, the proposed model and its SCAD surrogate are remarkably superior to the $\ell_p$-norm regularized models, and are comparable even superior to those sparsity constrained models with the true sparsity and the sign flip ratio as inputs.

This paper concerns the twice epi-differentiability and parabolic regularity of a class of non-amenable functions, the composition of a piecewise twice differentiable (PWTD) function and a parabolically semidifferentiable mapping. Such composite functions often appear in composite optimization problems, disjunctive optimization problems, and low-rank and/or sparsity optimization problems. By establishing the proper twice epi-differentiability and parabolic epi-differentiability of PWTD functions, we prove the parabolic epi-differentiability of this class of composite functions, and its twice epi-differentiability under the parabolic regularity assumption. Then, we identify a condition to ensure its parabolic regularity with the help of an upper and lower estimate of its second subderivative, and demonstrate that this condition holds for several classes of specific non-amenable functions., Comment: 32pages

In this paper, we consider a large class of nonlinear equations derived from first-order type methods for solving composite optimization problems. Traditional approaches to establishing superlinear convergence rates of semismooth Newton-type methods for solving nonlinear equations usually postulate either nonsingularity of the B-Jacobian or smoothness of the equation. We investigate the feasibility of both conditions. For the nonsingularity condition, we present equivalent characterizations in broad generality, and illustrate that they are easy-to-check criteria for some examples. For the smoothness condition, we show that it holds locally for a large class of residual mappings derived from composite optimization problems. Furthermore, we investigate a relaxed version of the smoothness condition - smoothness restricted to certain active manifolds. We present a conceptual algorithm utilizing such structures and prove that it has a superlinear convergence rate., Comment: 29 pages

This work extends the iterative framework proposed by Attouch et al. (in Math. Program. 137: 91-129, 2013) for minimizing a nonconvex and nonsmooth function $\Phi$ so that the generated sequence possesses a Q-superlinear convergence rate. This framework consists of a monotone decrease condition, a relative error condition and a continuity condition, and the first two conditions both involve a parameter $p\!>0$. We justify that any sequence conforming to this framework is globally convergent when $\Phi$ is a Kurdyka-{\L}ojasiewicz (KL) function, and the convergence has a Q-superlinear rate of order $\frac{p}{\theta(1+p)}$ when $\Phi$ is a KL function of exponent $\theta\in(0,\frac{p}{p+1})$. Then, we illustrate that the iterate sequence generated by an inexact $q\in[2,3]$-order regularization method for composite optimization problems with a nonconvex and nonsmooth term belongs to this framework, and consequently, first achieve the Q-superlinear convergence rate of order $4/3$ for an inexact cubic regularization method to solve this class of composite problems with KL property of exponent $1/2$.

This paper focuses on the minimization of a sum of a twice continuously differentiable function $f$ and a nonsmooth convex function. An inexact regularized proximal Newton method is proposed by an approximation to the Hessian of $f$ involving the $\varrho$th power of the KKT residual. For $\varrho=0$, we justify the global convergence of the iterate sequence for the KL objective function and its R-linear convergence rate for the KL objective function of exponent $1/2$. For $\varrho\in(0,1)$, by assuming that cluster points satisfy a locally H\"{o}lderian error bound of order $q$ on a second-order stationary point set and a local error bound of order $q>1\!+\!\varrho$ on the common stationary point set, respectively, we establish the global convergence of the iterate sequence and its superlinear convergence rate with order depending on $q$ and $\varrho$. A dual semismooth Newton augmented Lagrangian method is also developed for seeking an inexact minimizer of subproblems. Numerical comparisons with two state-of-the-art methods on $\ell_1$-regularized Student's $t$-regressions, group penalized Student's $t$-regressions, and nonconvex image restoration confirm the efficiency of the proposed method.

This paper is concerned with $\ell_q\,(0

This paper is concerned with a class of nonmonotone descent methods for minimizing a proper lower semicontinuous KL function $\Phi$, which generates a sequence satisfying a nonmonotone decrease condition and a relative error tolerance. Under suitable assumptions, we prove that the whole sequence converges to a limiting critical point of $\Phi$ and, when $\Phi$ is a KL function of exponent $\theta\in[0,1)$, the convergence admits a linear rate if $\theta\in[0,1/2]$ and a sublinear rate associated to $\theta$ if $\theta\in(1/2,1)$. The required assumptions are shown to be sufficient and necessary if $\Phi$ is also weakly convex on a neighborhood of stationary point set. Our results resolve the convergence problem on the iterate sequence generated by a class of nonmonotone line search algorithms for nonconvex and nonsmooth problems, and also extend the convergence results of monotone descent methods for KL optimization problems. As the applications, we achieve the convergence of the iterate sequence for the nonmonotone line search proximal gradient method with extrapolation and the nonmonotone line search proximal alternating minimization method with extrapolation. Numerical experiments are conducted for zero-norm and column $\ell_{2,0}$-norm regularized problems to validate their efficiency.

This paper is concerned with a class of optimization problems with the nonnegative orthogonal constraint, in which the objective function is $L$-smooth on an open set containing the Stiefel manifold ${\rm St}(n,r)$. We derive a locally Lipschitzian error bound for the feasible points without zero rows when $n>r>1$, and when $n>r=1$ or $n=r$ achieve a global Lipschitzian error bound. Then, we show that the penalty problem induced by the elementwise $\ell_1$-norm distance to the nonnegative cone is a global exact penalty, and so is the one induced by its Moreau envelope under a lower second-order calmness of the objective function. A practical penalty algorithm is developed by solving approximately a series of smooth penalty problems with a retraction-based nonmonotone line-search proximal gradient method, and any cluster point of the generated sequence is shown to be a stationary point of the original problem. Numerical comparisons with the ALM \citep{Wen13} and the exact penalty method \citep{JiangM22} indicate that our penalty method has an advantage in terms of the quality of solutions despite taking a little more time., Comment: 34 pages, and 6 figures

This paper is concerned with the calmness of a partial perturbation to the composite rank constraint system, an intersection of the rank constraint set and a general closed set, which is shown to be equivalent to a local Lipschitz-type error bound and also a global Lipschitz-type error bound under a certain compactness. Based on its lifted formulation, we derive two criteria for identifying those closed sets such that the associated partial perturbation possesses the calmness, and provide a collection of examples to demonstrate that the criteria are satisfied by common nonnegative and positive semidefinite rank constraint sets. Then, we use the calmness of this perturbation to obtain several global exact penalties for rank constrained optimization problems, and a family of equivalent DC surrogates for rank regularized problems.

This paper is concerned with the column $\ell_{2,0}$-regularized factorization model of low-rank matrix recovery problems and its computation. The column $\ell_{2,0}$-norm of factor matrices is introduced to promote column sparsity of factors and low-rank solutions. For this nonconvex discontinuous optimization problem, we develop an alternating majorization-minimization (AMM) method with extrapolation, and a hybrid AMM in which a majorized alternating proximal method is proposed to seek an initial factor pair with less nonzero columns and the AMM with extrapolation is then employed to minimize of a smooth nonconvex loss. We provide the global convergence analysis for the proposed AMM methods and apply them to the matrix completion problem with non-uniform sampling schemes. Numerical experiments are conducted with synthetic and real data examples, and comparison results with the nuclear-norm regularized factorization model and the max-norm regularized convex model show that the column $\ell_{2,0}$-regularized factorization model has an advantage in offering solutions of lower error and rank within less time.

This paper is concerned with the unconstrained binary polynomial program (UBPP), which has a host of applications in many science and engineering fields. By leveraging the global exact penalty for its DC constrained SDP reformulation, we achieve an equivalent DC penalized SDP, and propose a continuous relaxation approach by seeking the critical point of the Burer-Monteiro factorization for a finite number of DC penalized SDPs with increasing penalty factors. A globally convergent majorization-minimization (MM) method with extrapolation is also developed to capture such critical points. Under a mild condition, we show that the rank-one projection of the output for the relaxation approach is an approximate feasible solution of the UBPP and quantify the upper bound of its objective value from the optimal value. Numerical comparisons with the SDP relaxation method armed with a special random rounding technique and the DC relaxation approach based on the solution of linear SDPs confirm the efficiency of the proposed relaxation approach, which can solve the instance of \textbf{20000} variables in \textbf{15} minutes and yield an upper bound to the optimal value and the known best value with a relative error at most \textbf{1.824\%} and \textbf{2.870\%}, respectively.

This paper is concerned with a class of zero-norm regularized piecewise linear-quadratic (PLQ) composite minimization problems, which covers the zero-norm regularized $\ell_1$-loss minimization problem as a special case. For this class of nonconvex nonsmooth problems, we show that its equivalent MPEC reformulation is partially calm on the set of global optima and make use of this property to derive a family of equivalent DC surrogates. Then, we propose a proximal majorization-minimization (MM) method, a convex relaxation approach not in the DC algorithm framework, for solving one of the DC surrogates which is a semiconvex PLQ minimization problem involving three nonsmooth terms. For this method, we establish its global convergence and linear rate of convergence, and under suitable conditions show that the limit of the generated sequence is not only a local optimum but also a good critical point in a statistical sense. Numerical experiments are conducted with synthetic and real data for the proximal MM method with the subproblems solved by a dual semismooth Newton method to confirm our theoretical findings, and numerical comparisons with a convergent indefinite-proximal ADMM for the partially smoothed DC surrogate verify its superiority in the quality of solutions and computing time.

This paper is concerned with second-order optimality conditions for the mathematical program with semidefinite cone complementarity constraints (SDCMPCC).To achieve this goal, we first provide an exact characterization on the second-order tangent set to the semidefinite cone complementarity (SDCC) set in terms of the second-order directional derivative of the projection operator onto the SDCC set, and then use the second-order tangent set to the SDCC set to derive second-order necessary and sufficient conditions for SDCMPCC under suitable subregularity constraint qualifications. An application is also illustrated in characterizing the second-order necessary optimality condition for the semidefinite rank regularized problem., Comment: 26pages

This paper is concerned with the squared F(robenius)-norm regularized factorization form for noisy low-rank matrix recovery problems. Under a suitable assumption on the restricted condition number of the Hessian for the loss function, we derive an error bound to the true matrix for the non-strict critical points with rank not more than that of the true matrix. Then, for the squared F-norm regularized factorized least squares loss function, under the noisy and full sample setting we establish its KL property of exponent $1/2$ on its global minimizer set, and under the noisy and partial sample setting achieve this property for a class of critical points. These theoretical findings are also confirmed by solving the squared F-norm regularized factorization problem with an accelerated alternating minimization method.

This paper is concerned with the factorization form of the rank regularized loss minimization problem. To cater for the scenario in which only a coarse estimation is available for the rank of the true matrix, an $\ell_{2,0}$-norm regularized term is added to the factored loss function to reduce the rank adaptively; and account for the ambiguities in the factorization, a balanced term is then introduced. For the least squares loss, under a restricted condition number assumption on the sampling operator, we establish the KL property of exponent $1/2$ of the nonsmooth factored composite function and its equivalent DC reformulations in the set of their global minimizers. We also confirm the theoretical findings by applying a proximal linearized alternating minimization method to the regularized factorizations., Comment: 29 pages, 3 figures

This paper is concerned with the computation of the high-dimensional zero-norm penalized quantile regression estimator, defined as a global minimizer of the zero-norm penalized check loss function. To seek a desirable approximation to the estimator, we reformulate this NP-hard problem as an equivalent augmented Lipschitz optimization problem, and exploit its coupled structure to propose a multi-stage convex relaxation approach (MSCRA\_PPA), each step of which solves inexactly a weighted $\ell_1$-regularized check loss minimization problem with a proximal dual semismooth Newton method. Under a restricted strong convexity condition, we provide the theoretical guarantee for the MSCRA\_PPA by establishing the error bound of each iterate to the true estimator and the rate of linear convergence in a statistical sense. Numerical comparisons on some synthetic and real data show that MSCRA\_PPA not only has comparable even better estimation performance, but also requires much less CPU time.

For the rank regularized minimization problem, we introduce several kinds of stationary points by the problem itself and its equivalent reformulations including the mathematical program with an equilibrium constraint (MPEC), the global exact penalty of the MPEC,the surrogate yielded by eliminating the dual part in the exact penalty. A clear relation chart is established for these stationary points, which guides the user to choose an appropriate reformulation for seeking a low-rank solution. As a byproduct, we also provide a weaker condition for a local minimizer of the MPEC to be the M-stationary point by characterizing the directional limiting normal cone to the graph of the normal cone mapping of the positive semidefinite (PSD) cone., Comment: 28 pages

This paper focuses on the quadratic optimization over two classes of nonnegative zero-norm constraints: nonnegative zero-norm sphere constraint and zero-norm simplex constraint, which have important applications in nonnegative sparse eigenvalue problems and sparse portfolio problems, respectively. We establish the KL property of exponent 1/2 for the extended-valued objective function of these nonconvex and nonsmooth optimization problems, and use this crucial property to develop a globally and linearly convergent projection gradient descent (PGD) method. Numerical results are included for nonegative sparse principal component analysis and sparse portfolio problems with synthetic and real data to confirm the theoretical results., Comment: 4 figures. arXiv admin note: text overlap with arXiv:1811.04371 We have writen a new parper including most of this article, so we should withdraw it

For a proper extended real-valued function, this work focuses on the relationship between the subregularity of its subdifferential mapping relative to the critical set and its KL property of exponent 1/2. When the function is lsc convex, we establish the equivalence between them under the continuous assumption on the critical set. Then, for the uniformly prox-regular function, under its continuity on the local minimum set, the KL property of exponent 1/2 on the local minimum set is shown to be equivalent to the subregularity of its subdifferential relative to this set. Moreover, for this class of nonconvex functions, under a separation assumption of stationary values, we show that the subregularity of its subdifferential relative to the critical set also implies its KL property of exponent $1/2$. These results provide a bridge for the two kinds of regularity, and their application is illustrated by examples., Comment: 18 pages, 2 figures

This paper focuses on a class of zero-norm composite optimization problems. For this class of nonconvex nonsmooth problems, we establish the Kurdyka-Lojasiewicz property of exponent being a half for its objective function under a suitable assumption, and provide some examples to illustrate that such an assumption is not very restricted which, in particular, involve the zero-norm regularized or constrained piecewise linear-quadratic function, the zero-norm regularized or constrained logistic regression function, the zero-norm regularized or constrained quadratic function over a sphere.

Wasserstein barycenter is the centroid of a collection of discrete probability distributions which minimizes the average of the $\ell_2$-Wasserstein distance. This paper focuses on the computation of Wasserstein barycenters under the case where the support points are free, which is known to be a severe bottleneck in the D2-clustering due to the large-scale and nonconvexity. We develop an inexact proximal alternating minimization (iPAM) method for computing an approximate Wasserstein barycenter, and provide its global convergence analysis. This method can achieve a good accuracy with a reduced computational cost when the unknown support points of the barycenter have low cardinality. Numerical comparisons with the 3-block B-ADMM in \cite{YeWWL17} and an alternating minimization method involving the LP subproblems on synthetic and real data show that the proposed iPAM can yield comparable even a little better objective values in less CPU time, and hence the computed barycenter will render a better role in the D2-clustering.

This paper proposes a mechanism to produce equivalent Lipschitz surrogates for zero-norm and rank optimization problems by means of the global exact penalty for their equivalent mathematical programs with an equilibrium constraint (MPECs). Specifically, we reformulate these combinatorial problems as equivalent MPECs by the variational characterization of the zero-norm and rank function, show that their penalized problems, yielded by moving the equilibrium constraint into the objective, are the global exact penalization, and obtain the equivalent Lipschitz surrogates by eliminating the dual variable in the global exact penalty. These surrogates, including the popular SCAD function in statistics, are also difference of two convex functions (D.C.) if the function and constraint set involved in zero-norm and rank optimization problems are convex. We illustrate an application by designing a multi-stage convex relaxation approach to the rank plus zero-norm regularized minimization problem.

This paper concerns with the group zero-norm regularized least squares estimator which, in terms of the variational characterization of the zero-norm, can be obtained from a mathematical program with equilibrium constraints (MPEC). By developing the global exact penalty for the MPEC, this estimator is shown to arise from an exact penalization problem that not only has a favorable bilinear structure but also implies a recipe to deliver equivalent DC estimators such as the SCAD and MCP estimators. We propose a multi-stage convex relaxation approach (GEP-MSCRA) for computing this estimator, and under a restricted strong convexity assumption on the design matrix, establish its theoretical guarantees which include the decreasing of the error bounds for the iterates to the true coefficient vector and the coincidence of the iterates after finite steps with the oracle estimator. Finally, we implement the GEP-MSCRA with the subproblems solved by a semismooth Newton augmented Lagrangian method (ALM) and compare its performance with that of SLEP and MALSAR, the solvers for the weighted $\ell_{2,1}$-norm regularized estimator, on synthetic group sparse regression problems and real multi-task learning problems. Numerical comparison indicates that the GEP-MSCRA has significant advantage in reducing error and achieving better sparsity than the SLEP and the MALSAR do.

This paper is concerned with high-dimensional error-in-variables regression that aims at identifying a small number of important interpretable factors for corrupted data from many applications where measurement errors or missing data can not be ignored. Motivated by CoCoLasso due to Datta and Zou \cite{Datta16} and the advantage of the zero-norm regularized LS estimator over Lasso for clean data, we propose a calibrated zero-norm regularized LS (CaZnRLS) estimator by constructing a calibrated least squares loss with a positive definite projection of an unbiased surrogate for the covariance matrix of covariates, and use the multi-stage convex relaxation approach to compute the CaZnRLS estimator. Under a restricted eigenvalue condition on the true matrix of covariates, we derive the $\ell_2$-error bound of every iterate and establish the decreasing of the error bound sequence, and the sign consistency of the iterates after finite steps. The statistical guarantees are also provided for the CaZnRLS estimator under two types of measurement errors. Numerical comparisons with CoCoLasso and NCL (the nonconvex Lasso proposed by Poh and Wainwright \cite{Loh11}) demonstrate that CaZnRLS not only has the comparable or even better relative RSME but also has the least number of incorrect predictors identified.

This paper is concerned with the strong calmness of the KKT solution mapping for a class of canonically perturbed conic programming, which plays a central role in achieving fast convergence under situations when the Lagrange multiplier associated to a solution of these conic optimization problems is not unique. We show that the strong calmness of the KKT solution mapping is equivalent to a local error bound for solutions of perturbed KKT system, and is also equivalent to the pseudo-isolated calmness of the stationary point mapping along with the calmness of the multiplier set map at the corresponding reference point. Sufficient conditions are also provided for the strong calmness by establishing the pseudo-isolated calmness of the stationary point mapping in terms of the noncriticality of the associated multiplier, and the calmness of the multiplier set mapping in terms of a relative interior condition for the multiplier set. These results cover and extend the existing ones in \cite{Hager99,Izmailov12} for nonlinear programming and in \cite{Cui16,Zhang17} for semidefinite programming., Comment: 21 pages

This paper concerns with the graphical derivative of the normals to the conic constraint $g(x)\in\!K$, where $g\!:\mathbb{X}\to\mathbb{Y}$ is a twice continuously differentiable mapping and $K\subseteq\mathbb{Y}$ is a nonempty closed convex set assumed to be $C^2$-cone reducible. Such a generalized derivative plays a crucial role in characterizing isolated calmness of the solution maps to generalized equations whose multivalued parts are modeled via the normals to the nonconvex set $\Gamma=g^{-1}(K)$. The main contribution of this paper is to provide an exact characterization for the graphical derivative of the normals to this class of nonconvex conic constraints under an assumption without requiring the nondegeneracy of the reference point as the papers \cite{Gfrerer17,Mordu15,Mordu151} do., Comment: 28pages

This paper concerns with a noisy structured low-rank matrix recovery problem which can be modeled as a structured rank minimization problem. We reformulate this problem as a mathematical program with a generalized complementarity constraint (MPGCC), and show that its penalty version, yielded by moving the generalized complementarity constraint to the objective, has the same global optimal solution set as the MPGCC does whenever the penalty parameter is over a threshold. Then, by solving the exact penalty problem in an alternating way, we obtain a multi-stage convex relaxation approach. We provide theoretical guarantees for our approach under a mild restricted eigenvalue condition, by quantifying the reduction of the error and approximate rank bounds of the first stage convex relaxation (which is exactly the nuclear norm relaxation) in the subsequent stages and establishing the geometric convergence of the error sequence in a statistical sense. Numerical experiments are conducted for some structured low-rank matrix recovery examples to confirm our theoretical findings., Comment: 29 pages, 2 figures

In this paper we establish an $\mathcal{O}({1}/{k})$ weighted iteration complexity on the KKT residuals yielded by the sPADMM (semi-proximal alternating direction method of multiplier) for the convex composite optimization problem. This result, which is derived with the help of a novel generalized HPE (hybrid proximal extra-gradient) iteration formula, first fills the gap on the ergodic iteration complexity of the classic ADMM with a large step-size and its many proximal variants.