Search

Given labeled points in a high-dimensional vector space, we seek a low-dimensional subspace such that projecting onto this subspace maintains some prescribed distance between points of differing labels. Intended applications include compressive classification. Taking inspiration from large margin nearest neighbor classification, this paper introduces a semidefinite relaxation of this problem. Unlike its predecessors, this relaxation is amenable to theoretical analysis, allowing us to provably recover a planted projection operator from the data.

Gerrymandering is a long-standing issue within the U.S. political system, and it has received scrutiny recently by the U.S. Supreme Court. In this note, we prove that deciding whether there exists a fair redistricting among legal maps is NP-hard. To make this precise, we use simplified notions of "legal" and "fair" that account for desirable traits such as geographic compactness of districts and sufficient representation of voters. The proof of our result is inspired by the work of Mahanjan, Minbhorkar and Varadarajan that proves that planar k-means is NP-hard.

We study lines through the origin of finite-dimensional complex vector spaces that enjoy a doubly transitive automorphism group. In doing so, we make fundamental connections with both discrete geometry and algebraic combinatorics. In particular, we show that doubly transitive lines are necessarily optimal packings in complex projective space, and we introduce a fruitful generalization of regular abelian distance-regular antipodal covers of the complete graph.

It has been experimentally established that deep neural networks can be used to produce good generative models for real world data. It has also been established that such generative models can be exploited to solve classical inverse problems like compressed sensing and super resolution. In this work we focus on the classical signal processing problem of image denoising. We propose a theoretical setting that uses spherical harmonics to identify what mathematical properties of the activation functions will allow signal denoising with local methods.

Bizarrely shaped voting districts are frequently lambasted as likely instances of gerrymandering. In order to systematically identify such instances, researchers have devised several tests for so-called geographic compactness (i.e., shape niceness). We demonstrate that under certain conditions, a party can gerrymander a competitive state into geographically compact districts to win an average of over 70% of the districts. Our results suggest that geometric features alone may fail to adequately combat partisan gerrymandering., Comment: 12 pages, 2 figures

An equiangular tight frame (ETF) is a type of optimal packing of lines in Euclidean space. A regular simplex is a special type of ETF in which the number of vectors is one more than the dimension of the space they span. In this paper, we consider ETFs that contain a regular simplex, that is, have the property that a subset of its vectors forms a regular simplex. As we explain, such ETFs are characterized as those that achieve equality in a certain well-known bound from the theory of compressed sensing. We then consider the so-called binder of such an ETF, namely the set of all regular simplices that it contains. We provide a new algorithm for computing this binder in terms of products of entries of the ETF's Gram matrix. In certain circumstances, we show this binder can be used to produce a particularly elegant Naimark complement of the corresponding ETF. Other times, an ETF is a disjoint union of regular simplices, and we show this leads to a certain type of optimal packing of subspaces known as an equichordal tight fusion frame. We conclude by considering the extent to which these ideas can be applied to numerous known constructions of ETFs, including harmonic ETFs.

The U.S. Supreme Court is currently deliberating over whether a proposed mathematical formula should be used to detect unconstitutional partisan gerrymandering. We show that in some cases, this formula will only flag bizarrely shaped districts as potentially constitutional., Comment: 7 pages, 1 figure

Efficient algorithms for $k$-means clustering frequently converge to suboptimal partitions, and given a partition, it is difficult to detect $k$-means optimality. In this paper, we develop an a posteriori certifier of approximate optimality for $k$-means clustering. The certifier is a sub-linear Monte Carlo algorithm based on Peng and Wei's semidefinite relaxation of $k$-means. In particular, solving the relaxation for small random samples of the dataset produces a high-confidence lower bound on the $k$-means objective, and being sub-linear, our algorithm is faster than $k$-means++ when the number of data points is large. We illustrate the performance of our algorithm with both numerical experiments and a performance guarantee: If the data points are drawn independently from any mixture of two Gaussians over $\mathbb{R}^m$ with identity covariance, then with probability $1-O(1/m)$, our $\operatorname{poly}(m)$-time algorithm produces a 3-approximation certificate with 99% confidence., Comment: 8 pages

We provide a general program for finding nice arrangements of points in real or complex projective space from transitive actions of finite groups. In many cases, these arrangements are optimal in the sense of maximizing the minimum distance. We introduce our program in terms of general Schurian association schemes before focusing on the special case of Gelfand pairs. Notably, our program unifies a variety of existing packings with heretofore disparate constructions. In addition, we leverage our program to construct the first known infinite family of equiangular lines with Heisenberg symmetry., Comment: 31 pages

Equi-chordal and equi-isoclinic tight fusion frames (ECTFFs and EITFFs) are both types of optimal packings of subspaces in Euclidean spaces. In the special case where these subspaces are one-dimensional, ECTFFs and EITFFs both correspond to types of optimal packings of lines known as equiangular tight frames. In this brief note, we review some of the fundamental ideas and results concerning ECTFFs and EITFFs.

This paper applies techniques from algebraic and differential geometry to determine how to best pack points in real projective spaces. We present a computer-assisted proof of the optimality of a particular 6-packing in $\mathbb{R}\mathbf{P}^3$, we introduce a linear-time constant-factor approximation algorithm for packing in the so-called Gerzon range, and we provide local optimality certificates for two infinite families of packings. Finally, we present perfected versions of various putatively optimal packings from Sloane's online database, along with a handful of infinite families they suggest, and we prove that these packings enjoy a certain weak notion of optimality., Comment: 31 pages, 2 figures

An equiangular tight frame (ETF) is a type of optimal packing of lines in Euclidean space. They are often represented as the columns of a short, fat matrix. In certain applications we want this matrix to be flat, that is, have the property that all of its entries have modulus one. In particular, real flat ETFs are equivalent to self-complementary binary codes that achieve the Grey-Rankin bound. Some flat ETFs are (complex) Hadamard ETFs, meaning they arise by extracting rows from a (complex) Hadamard matrix. These include harmonic ETFs, which are obtained by extracting the rows of a character table that correspond to a difference set in the underlying finite abelian group. In this paper, we give some new results about flat ETFs. One of these results gives an explicit Naimark complement for all Steiner ETFs, which in turn implies that all Kirkman ETFs are possibly-complex Hadamard ETFs. This in particular produces a new infinite family of real flat ETFs. Another result establishes an equivalence between real flat ETFs and certain types of quasi-symmetric designs, resulting in a new infinite family of such designs.

We use group schemes to construct optimal packings of lines through the origin. In this setting, optimal line packings are naturally characterized using representation theory, which in turn leads to a necessary integrality condition for the existence of equiangular central group frames. We conclude with an infinite family of optimal line packings using the group schemes associated with certain Suzuki 2-groups, specifically, extensions of Heisenberg groups. Notably, this is the first known infinite family of equiangular tight frames generated by representations of nonabelian groups., Comment: 28 pages. During the revision process, portions of v1 were removed and expanded to form arXiv:1709.03558

An equiangular tight frame (ETF) is a type of optimal packing of lines in a finite-dimensional Hilbert space. ETFs arise in various applications, such as waveform design for wireless communication, compressed sensing, quantum information theory and algebraic coding theory. In a recent paper, signature matrices of ETFs were constructed from abelian distance regular covers of complete graphs. We extend this work, constructing ETF synthesis operators from abelian generalized quadrangles, and vice versa. This produces a new infinite family of complex ETFs as well as a new proof of the existence of certain generalized quadrangles. This work involves designing matrices whose entries are polynomials over a finite abelian group. As such, it is related to the concept of a polyphase matrix of a finite filter bank.

We introduce a model-free relax-and-round algorithm for k-means clustering based on a semidefinite relaxation due to Peng and Wei. The algorithm interprets the SDP output as a denoised version of the original data and then rounds this output to a hard clustering. We provide a generic method for proving performance guarantees for this algorithm, and we analyze the algorithm in the context of subgaussian mixture models. We also study the fundamental limits of estimating Gaussian centers by k-means clustering in order to compare our approximation guarantee to the theoretically optimal k-means clustering solution.

An equiangular tight frame (ETF) is a set of equal norm vectors in a Euclidean space whose coherence is as small as possible, equaling the Welch bound. Also known as Welch-bound-equality sequences, such frames arise in various applications, such as waveform design, quantum information theory, compressed sensing and algebraic coding theory. ETFs seem to be rare, and only a few methods of constructing them are known. In this paper, we present a new infinite family of complex ETFs that arises from hyperovals in finite projective planes. In particular, we give the first ever construction of a complex ETF of 76 vectors in a space of dimension 19. Recently, a computer-assisted approach was used to show that real ETFs of this size do not exist, resolving a longstanding open problem in this field. Our construction is a modification of a previously known technique for constructing ETFs from balanced incomplete block designs.

Equiangular tight frames provide optimal packings of lines through the origin. We combine Steiner triple systems with Hadamard matrices to produce a new infinite family of equiangular tight frames. This in turn leads to new constructions of strongly regular graphs and distance-regular antipodal covers of the complete graph., Comment: 11 pages

Recently, Bandeira [arXiv:1509.00824] introduced a new type of algorithm (the so-called probably certifiably correct algorithm) that combines fast solvers with the optimality certificates provided by convex relaxations. In this paper, we devise such an algorithm for the problem of k-means clustering. First, we prove that Peng and Wei's semidefinite relaxation of k-means is tight with high probability under a distribution of planted clusters called the stochastic ball model. Our proof follows from a new dual certificate for integral solutions of this semidefinite program. Next, we show how to test the optimality of a proposed k-means solution using this dual certificate in quasilinear time. Finally, we analyze a version of spectral clustering from Peng and Wei that is designed to solve k-means in the case of two clusters. In particular, we show that this quasilinear-time method typically recovers planted clusters under the stochastic ball model., Comment: Major revision from previous version. This paper is a extension of and improvement to the authors' preprint [arXiv:1505.04778]

An equiangular tight frame (ETF) is a set of unit vectors whose coherence achieves the Welch bound, and so is as incoherent as possible. Though they arise in many applications, only a few methods for constructing them are known. Motivated by the connection between real ETFs and graph theory, we introduce the notion of ETFs that are symmetric about their centroid. We then discuss how well-known constructions, such as harmonic ETFs and Steiner ETFs, can have centroidal symmetry. Finally, we establish a new equivalence between centroid-symmetric real ETFs and certain types of strongly regular graphs (SRGs). Together, these results give the first proof of the existence of certain SRGs, as well as the disproofs of the existence of others.

This paper discusses the theory and application of learning Boolean functions that are concentrated in the Fourier domain. We first estimate the VC dimension of this function class in order to establish a small sample complexity of learning in this case. Next, we propose a computationally efficient method of empirical risk minimization, and we apply this method to the MNIST database of handwritten digits. These results demonstrate the effectiveness of our model for modern classification tasks. We conclude with a list of open problems for future investigation.

Recently, Awasthi et al. introduced an SDP relaxation of the $k$-means problem in $\mathbb R^m$. In this work, we consider a random model for the data points in which $k$ balls of unit radius are deterministically distributed throughout $\mathbb R^m$, and then in each ball, $n$ points are drawn according to a common rotationally invariant probability distribution. For any fixed ball configuration and probability distribution, we prove that the SDP relaxation of the $k$-means problem exactly recovers these planted clusters with probability $1-e^{-\Omega(n)}$ provided the distance between any two of the ball centers is $>2+\epsilon$, where $\epsilon$ is an explicit function of the configuration of the ball centers, and can be arbitrarily small when $m$ is large.

We develop new discrete uncertainty principles in terms of numerical sparsity, which is a continuous proxy for the 0-norm. Unlike traditional sparsity, the continuity of numerical sparsity naturally accommodates functions which are nearly sparse. After studying these principles and the functions that achieve exact or near equality in them, we identify certain consequences in a number of sparse signal processing applications.

A Grassmannian frame is a collection of unit vectors which are optimally incoherent. To date, the vast majority of explicit Grassmannian frames are equiangular tight frames (ETFs). This paper surveys every known construction of ETFs and tabulates existence for sufficiently small dimensions.

We consider the classical 1D phase retrieval problem. In order to overcome the difficulties associated with phase retrieval from measurements of the Fourier magnitude, we treat recovery from the magnitude of the short-time Fourier transform (STFT). We first show that the redundancy offered by the STFT enables unique recovery for arbitrary nonvanishing inputs, under mild conditions. An efficient algorithm for recovery of a sparse input from the STFT magnitude is then suggested, based on an adaptation of the recently proposed GESPAR algorithm. We demonstrate through simulations that using the STFT leads to improved performance over recovery from the oversampled Fourier magnitude with the same number of measurements., Comment: To appear in IEEE Signal Processing Letters

We study the restricted isometry property of a matrix that is built from the discrete Fourier transform matrix by collecting rows indexed by quadratic residues. We find an $\epsilon>0$ such that, conditioned on a folklore conjecture in number theory, this matrix satisfies the restricted isometry property with sparsity parameter $K=\Omega(M^{1/2+\epsilon})$, where $M$ is the number of rows., Comment: 6 pages

Compressed sensing seeks to invert an underdetermined linear system by exploiting additional knowledge of the true solution. Over the last decade, several instances of compressed sensing have been studied for various applications, and for each instance, reconstruction guarantees are available provided the sensing operator satisfies certain sufficient conditions. In this paper, we completely characterize the sensing operators which allow uniformly stable and robust reconstruction by convex optimization for many of these instances. The characterized sensing operators satisfy a new property we call the robust width property, which simultaneously captures notions of widths from approximation theory and of restricted eigenvalues from statistical regression. We provide a geometric interpretation of this property, we discuss its relationship with the restricted isometry property, and we apply techniques from geometric functional analysis to find random matrices which satisfy the property with high probability., Comment: 24 pages

Subspace-based signal processing traditionally focuses on problems involving a few subspaces. Recently, a number of problems in different application areas have emerged that involve a significantly larger number of subspaces relative to the ambient dimension. It becomes imperative in such settings to first identify a smaller set of active subspaces that contribute to the observation before further processing can be carried out. This problem of identification of a small set of active subspaces among a huge collection of subspaces from a single (noisy) observation in the ambient space is termed subspace unmixing. This paper formally poses the subspace unmixing problem under the parsimonious subspace-sum (PS3) model, discusses connections of the PS3 model to problems in wireless communications, hyperspectral imaging, high-dimensional statistics and compressed sensing, and proposes a low-complexity algorithm, termed marginal subspace detection (MSD), for subspace unmixing. The MSD algorithm turns the subspace unmixing problem for the PS3 model into a multiple hypothesis testing (MHT) problem and its analysis in the paper helps control the family-wise error rate of this MHT problem at any level $\alpha \in [0,1]$ under two random signal generation models. Some other highlights of the analysis of the MSD algorithm include: (i) it is applicable to an arbitrary collection of subspaces on the Grassmann manifold; (ii) it relies on properties of the collection of subspaces that are computable in polynomial time; and ($iii$) it allows for linear scaling of the number of active subspaces as a function of the ambient dimension. Finally, numerical results are presented in the paper to better understand the performance of the MSD algorithm., Comment: Submitted for journal publication; 33 pages, 14 figures

The restricted isometry property (RIP) is an important matrix condition in compressed sensing, but the best matrix constructions to date use randomness. This paper leverages pseudorandom properties of the Legendre symbol to reduce the number of random bits in an RIP matrix with Bernoulli entries. In this regard, the Legendre symbol is not special---our main result naturally generalizes to any small-bias sample space. We also conjecture that no random bits are necessary for our Legendre symbol--based construction.

This paper addresses the fundamental question of when convex sets remain disjoint after random projection. We provide an analysis using ideas from high-dimensional convex geometry. For ellipsoids, we provide a bound in terms of the distance between these ellipsoids and simple functions of their polynomial coefficients. As an application, this theorem provides bounds for compressive classification of convex sets. Rather than assuming that the data to be classified is sparse, our results show that the data can be acquired via very few measurements yet will remain linearly separable. We demonstrate the feasibility of this approach in the context of hyperspectral imaging.

Matrices with the restricted isometry property (RIP) are of particular interest in compressed sensing. To date, the best known RIP matrices are constructed using random processes, while explicit constructions are notorious for performing at the "square-root bottleneck," i.e., they only accept sparsity levels on the order of the square root of the number of measurements. The only known explicit matrix which surpasses this bottleneck was constructed by Bourgain, Dilworth, Ford, Konyagin and Kutzarova. This chapter provides three contributions to further the groundbreaking work of Bourgain et al.: (i) we develop an intuition for their matrix construction and underlying proof techniques; (ii) we prove a generalized version of their main result; and (iii) we apply this more general result to maximize the extent to which their matrix construction surpasses the square-root bottleneck., Comment: Book chapter, submitted to Compressed Sensing and its Applications

Consider a scenario in which an unknown signal is transformed by a known linear operator, and then the pointwise absolute value of the unknown output function is reported. This scenario appears in several applications, and the goal is to recover the unknown signal -- this is called phase retrieval. Phase retrieval has been a popular subject of research in the last few years, both in determining whether complete information is available with a given linear operator, and in finding efficient and stable phase retrieval algorithms in the cases where complete information is available. Interestingly, there are a few ways to measure information completeness, and each way appears to be governed by a phase transition of sorts. This chapter will survey the state of the art with some of these phase transitions, and identify a few open problems for further research., Comment: Book chapter, survey of recent literature, submitted to Excursions in Harmonic Analysis: The February Fourier Talks at the Norbert Wiener Center

In this paper, we settle a long-standing problem on the connectivity of spaces of finite unit norm tight frames (FUNTFs), essentially affirming a conjecture first appearing in [Dykema and Strawn, 2003]. Our central technique involves continuous liftings of paths from the polytope of eigensteps to spaces of FUNTFs. After demonstrating this connectivity result, we refine our analysis to show that the set of nonsingular points on these spaces is also connected, and we use this result to show that spaces of FUNTFs are irreducible in the algebro-geometric sense, and also that generic FUNTFs are full spark., Comment: 33 pages, 4 figures

In many areas of imaging science, it is difficult to measure the phase of linear measurements. As such, one often wishes to reconstruct a signal from intensity measurements, that is, perform phase retrieval. In several applications the signal in question is believed to be sparse. In this paper, we use ideas from the recently developed polarization method for phase retrieval and provide an algorithm that is guaranteed to recover a sparse signal from a number of phaseless linear measurements that scales linearly with the sparsity of the signal (up to logarithmic factors). This is particularly remarkable since it is known that a certain popular class of convex methods is not able to perform recovery unless the number of measurements scales with the square of the sparsity of the signal. This is a shorter version of a more complete publication that will appear elsewhere.

In many applications, signals are measured according to a linear process, but the phases of these measurements are often unreliable or not available. To reconstruct the signal, one must perform a process known as phase retrieval. This paper focuses on completely determining signals with as few intensity measurements as possible, and on efficient phase retrieval algorithms from such measurements. For the case of complex M-dimensional signals, we construct a measurement ensemble of size 4M-4 which yields injective intensity measurements; this is conjectured to be the smallest such ensemble. For the case of real signals, we devise a theory of "almost" injective intensity measurements, and we characterize such ensembles. Later, we show that phase retrieval from M+1 almost injective intensity measurements is NP-hard, indicating that computationally efficient phase retrieval must come at the price of measurement redundancy., Comment: 18 pages, 1 figure

An equiangular tight frame (ETF) is a set of unit vectors in a Euclidean space whose coherence is as small as possible, equaling the Welch bound. Also known as Welch-bound-equality sequences, such frames arise in various applications, such as waveform design and compressed sensing. At the moment, there are only two known flexible methods for constructing ETFs: harmonic ETFs are formed by carefully extracting rows from a discrete Fourier transform; Steiner ETFs arise from a tensor-like combination of a combinatorial design and a regular simplex. These two classes seem very different: the vectors in harmonic ETFs have constant amplitude, whereas Steiner ETFs are extremely sparse. We show that they are actually intimately connected: a large class of Steiner ETFs can be unitarily transformed into constant-amplitude frames, dubbed Kirkman ETFs. Moreover, we show that an important class of harmonic ETFs is a subset of an important class of Kirkman ETFs. This connection informs the discussion of both types of frames: some Steiner ETFs can be transformed into constant-amplitude waveforms making them more useful in waveform design; some harmonic ETFs have low spark, making them less desirable for compressed sensing. We conclude by showing that real-valued constant-amplitude ETFs are equivalent to binary codes that achieve the Grey-Rankin bound, and then construct such codes using Kirkman ETFs.

In diffraction imaging, one is tasked with reconstructing a signal from its power spectrum. To resolve the ambiguity in this inverse problem, one might invoke prior knowledge about the signal, but phase retrieval algorithms in this vein have found limited success. One alternative is to create redundancy in the measurement process by illuminating the signal multiple times, distorting the signal each time with a different mask. Despite several recent advances in phase retrieval, the community has yet to construct an ensemble of masks which uniquely determines all signals and admits an efficient reconstruction algorithm. In this paper, we leverage the recently proposed polarization method to construct such an ensemble. We also present numerical simulations to illustrate the stability of the polarization method in this setting. In comparison to a state-of-the-art phase retrieval algorithm known as PhaseLift, we find that polarization is much faster with comparable stability., Comment: 18 pages, 3 figures

Recent advances in convex optimization have led to new strides in the phase retrieval problem over finite-dimensional vector spaces. However, certain fundamental questions remain: What sorts of measurement vectors uniquely determine every signal up to a global phase factor, and how many are needed to do so? Furthermore, which measurement ensembles lend stability? This paper presents several results that address each of these questions. We begin by characterizing injectivity, and we identify that the complement property is indeed a necessary condition in the complex case. We then pose a conjecture that 4M-4 generic measurement vectors are both necessary and sufficient for injectivity in M dimensions, and we prove this conjecture in the special cases where M=2,3. Next, we shift our attention to stability, both in the worst and average cases. Here, we characterize worst-case stability in the real case by introducing a numerical version of the complement property. This new property bears some resemblance to the restricted isometry property of compressed sensing and can be used to derive a sharp lower Lipschitz bound on the intensity measurement mapping. Localized frames are shown to lack this property (suggesting instability), whereas Gaussian random measurements are shown to satisfy this property with high probability. We conclude by presenting results that use a stochastic noise model in both the real and complex cases, and we leverage Cramer-Rao lower bounds to identify stability with stronger versions of the injectivity characterizations., Comment: 22 pages