Search

In this paper, we design the precoding matrix with broad coverage for massive multi-input multi-output public channels. In order to guarantee the efficient use of power amplifiers and improve the achievable ergodic rate, equal transmit power per-antenna and semi-unitary constraints on the precoding matrix are considered simultaneously. Within the framework of manifold optimization, the precoding matrix design under the above two constraints becomes an optimization problem over the intersection of the oblique manifold and the Stiefel manifold. We propose to use the steepest descent method on the intersection of these two manifolds to obtain the optimal solution. By using the alternating projections method, the search direction of the steepest descent method is derived. Meanwhile, the convergence analysis for the proposed approach is also provided. The proposed approach can be used for both omnidirectional and sector-shaped power pattern design. Simulation results show that the power pattern of our designed precoder has less variation in different spatial directions within the cell coverage compared with the existing Zadoff–Chu scheme. [ABSTRACT FROM AUTHOR]

When optimizing on a Riemannian manifold, it is important to use an efficient retraction, which maps a point on a tangent space to a point on the manifold. In this paper, we prove a map based on the QR factorization to be a retraction on the generalized Stiefel manifold. In addition, we propose an efficient implementation of the retraction based on the Cholesky QR factorization. Numerical experiments show that the proposed retraction is more efficient than the existing one based on the polar factorization. [ABSTRACT FROM AUTHOR]

In this paper, we consider a class of optimization problems with orthogonality constraints, the feasible region of which is called the Stiefel manifold. Our new framework combines a function value reduction step with a correction step. Different from the existing approaches, the function value reduction step of our algorithmic framework searches along the standard Euclidean descent directions instead of the vectors in the tangent space of the Stiefel manifold, and the correction step further reduces the function value and guarantees a symmetric dual variable at the same time. We construct two types of algorithms based on this new framework. The first type is based on gradient reduction including the gradient re ection (GR) and the gradient projection (GP) algorithms. The other one adopts a columnwise block coordinate descent (CBCD) scheme with a novel idea for solving the corresponding CBCD subproblem inexactly. We prove that both GR/GP with a fixed step size and CBCD belong to our algorithmic framework, and any clustering point of the iterates generated by the proposed framework is a first-order stationary point. Preliminary experiments illustrate that our new framework is of great potential. [ABSTRACT FROM AUTHOR]

In domain adaptation, the goal is to find common ground between two, potentially differently distributed, data sets. By finding common concepts present in two sets of words pertaining to different domains, one could leverage the performance of a classifier for one domain for use on the other domain. We propose a solution to the domain adaptation task, by efficiently solving an optimization problem through Stochastic Gradient Descent. We provide update rules that allow us to run Stochastic Gradient Descent directly on a matrix manifold: the steps compel the solution to stay on the Stiefel manifold. This manifold encompasses projection matrices of word vectors onto low-dimensional latent feature representations, which allows us to interpret the results: the rotation magnitude of the word vector projection for a given word corresponds to the importance of that word towards making the adaptation. Beyond this interpretability benefit, experiments show that the Stiefel manifold method performs better than state-of-the-art methods. [ABSTRACT FROM AUTHOR]

This article proposes new conjugate gradient method for unconstrained optimization by applying the Powell symmetrical technique in a defined sense. Using the Wolfe line search conditions, the global convergence property of the method is also obtained based on the spectral analysis of the conjugate gradient iteration matrix and the Zoutendijk condition for steepest descent methods. Preliminary numerical results for a set of 86 unconstrained optimization test problems verify the performance of the algorithm and show that the Generalized Descent Symmetrical Hestenes-Stiefel algorithm is competitive with the Fletcher-Reeves (FR) and Polak-Ribiére-Polyak (PRP+) algorithms. [ABSTRACT FROM AUTHOR]

This paper considers optimization problems on the Stiefel manifold $$X^{\mathsf{T}}X=I_p$$ , where $$X\in \mathbb {R}^{n \times p}$$ is the variable and $$I_p$$ is the $$p$$ -by- $$p$$ identity matrix. A framework of constraint preserving update schemes is proposed by decomposing each feasible point into the range space of $$X$$ and the null space of $$X^{\mathsf{T}}$$ . While this general framework can unify many existing schemes, a new update scheme with low complexity cost is also discovered. Then we study a feasible Barzilai-Borwein-like method under the new update scheme. The global convergence of the method is established with an adaptive nonmonotone line search. The numerical tests on the nearest low-rank correlation matrix problem, the Kohn-Sham total energy minimization and a specific problem from statistics demonstrate the efficiency of the new method. In particular, the new method performs remarkably well for the nearest low-rank correlation matrix problem in terms of speed and solution quality and is considerably competitive with the widely used SCF iteration for the Kohn-Sham total energy minimization. [ABSTRACT FROM AUTHOR]

Convex optimization methods are used for many machine learning models such as support vector machine. However, the requirement of a convex formulation can place limitations on machine learning models. In recent years, a number of machine learning methods not requiring convexity have emerged. In this paper, we study non-convex optimization problems on the Stiefel manifold in which the feasible set consists of a set of rectangular matrices with orthonormal column vectors. We present examples of non-convex optimization problems in machine learning and apply three nonlinear optimization methods for finding a local optimal solution; geometric gradient descent method, augmented Lagrangian method of multipliers, and alternating direction method of multipliers. Although the geometric gradient method is often used to solve non-convex optimization problems on the Stiefel manifold, we show that the alternating direction method of multipliers generally produces higher quality numerical solutions within a reasonable computation time. [ABSTRACT FROM AUTHOR]

In this paper, the integer programming problem is studied. We introduce the notion of integer basis and show that a given integer set can be converted into a fixed number of linear combinations of the basis elements. By employing the Stiefel manifold and optimal control theory, the combinatorial optimization problem can be converted into a continuous optimization problem over the continuous Stiefel manifold. As a result, gradient descent methods can be applied to find the optimal integer solution. We demonstrate by numerical examples that this approach can obtain good solutions. Furthermore, this method gives new insights into continuous relaxation for solving integer programming problems. [ABSTRACT FROM AUTHOR]

Mathematical programming is a rich and well-developed area in operations research. Nevertheless, there remain many challenging problems in this area, one of which is the large-scale optimization problem. In this article, a modified Hestenes and Stiefel (HS) conjugate gradient (CG) algorithm with a nonmonotone line search technique is presented. This algorithm possesses information about not only the gradient value but also the function value. Moreover, the sufficient descent condition holds without any line search. The global convergence is established for nonconvex functions under suitable conditions. Numerical results show that the proposed algorithm is advantageous to existing CG methods for large-scale optimization problems. [ABSTRACT FROM AUTHOR]

We study the problem of finding the global minimum of a homogeneous quadratic function of special kind over the Stiefel manifold. For two variants of this problem, a low bound is proposed that is the dual Lagrange bound in the quadratic statement obtained using a family of redundant restrictions. The dual bound is proved to be exact in the case where the problem is considered over Boolean variables. [ABSTRACT FROM AUTHOR]

This letter aims at illustrating the relevance of numerical integration of learning differential equations on differential manifolds. In particular, the task of learning with orthonormality constraints is dealt with, which is naturally formulated as an optimization task with the compact Stiefel manifold as neural parameter space. Intrinsic properties of the derived learning algorithms, such as stability and constraints preservation, are illustrated through experiments on minor and independent component analysis (ICA). [ABSTRACT FROM AUTHOR]

The optimization on Stiefel manifolds was discussed by Rapcsak in [1-3]. Here, some methods of global optimization are dealt with and tested on Stiefel manifolds. The structure of the optimizer points of a quadratic problem is studied theoretically and numerically for the lowest interesting dimensional case, as well as the criterion for the finiteness of the number of optimizer points. Then possible reduction tricks are examined together with a numerical study. In the computational investigation we also focused on a special case of the problem, namely when the coefficient matrices in the objective function are diagonal. [ABSTRACT FROM AUTHOR]

Presents novel algorithms that converge to a local minimum of a real-valued function f(X) subject to the constraint that the columns of the complex-valued matrix X are mutually orthogonal and have unit norm. Formulae for calculating the critical point of a quadratic function; Introduction of the complex Stiefel manifold; Optimization on the complex Grassmann manifold.