Search

In this paper, without requiring any constraint qualifications, we establish tight error bounds for the log-determinant cone, which is the closure of the hypograph of the perspective function of the log-determinant function. This error bound is obtained using the recently developed framework based on one-step facial residual functions., Comment: 32 pages, comments welcome

Error bounds are a requisite for trusting or distrusting solutions in an informed way. Until recently, provable error bounds in the absence of constraint qualifications were unattainable for many classes of cones that do not admit projections with known succinct expressions. We build such error bounds for the generalized power cones, using the recently developed framework of one-step facial residual functions. We also show that our error bounds are tight in the sense of that framework. Besides their utility for understanding solution reliability, the error bounds we discover have additional applications to the algebraic structure of the underlying cone, which we describe. In particular we use the error bounds to compute the dimension of the automorphism group for the generalized power cones, and to identify a set of generalized power cones that are self-dual, irreducible, nonhomogeneous, and perfect, Comment: 24 pages, title change, some minor fixes throughout the paper and removed the appendix. Comments welcome

We prove tight H\"olderian error bounds for all $p$-cones. Surprisingly, the exponents differ in several ways from those that have been previously conjectured; moreover, they illuminate $p$-cones as a curious example of a class of objects that possess properties in 3 dimensions that they do not in 4 or more. Using our error bounds, we analyse least squares problems with $p$-norm regularization, where our results enable us to compute the corresponding KL exponents for previously inaccessible values of $p$. Another application is a (relatively) simple proof that most $p$-cones are neither self-dual nor homogeneous. Our error bounds are obtained under the framework of facial residual functions, and we expand it by establishing for general cones an optimality criterion under which the resulting error bound must be tight., Comment: 37 pages, comments welcome. To appear at Mathematics of Operations Research

Friedlander, Mac\^{e}do, and Pong recently introduced the projected polar proximal point algorithm (P4A) for solving optimization problems by using the closed perspective transforms of convex objectives. We analyse a generalization (GP4A) which replaces the closed perspective transform with a more general closed gauge. We decompose GP4A into the iterative application of two separate operators, and analyse it as a splitting method. By showing that GP4A and its under-relaxations exhibit global convergence whenever a fixed point exists, we obtain convergence guarantees for P4A by letting the gauge specify to the closed perspective transform for a convex function. We then provide easy-to-verify sufficient conditions for the existence of fixed points for the GP4A, using the Minkowski function representation of the gauge. Conveniently, the approach reveals that global minimizers of the objective function for P4A form an exposed face of the dilated fundamental set of the closed perspective transform.

Every maximally monotone operator can be associated with a family of convex functions, called the Fitzpatrick family or family of representative functions. Surprisingly, in 2017, Burachik and Mart\'inez-Legaz showed that the well-known Bregman distance is a particular case of a general family of distances, each one induced by a specific maximally monotone operator and a specific choice of one of its representative functions. For the family of generalized Bregman distances, sufficient conditions for convexity, coercivity, and supercoercivity have recently been furnished. Motivated by these advances, we introduce in the present paper the generalized left and right envelopes and proximity operators, and we provide asymptotic results for parameters. Certain results extend readily from the more specific Bregman context, while others only extend for certain generalized cases. To illustrate, we construct examples from the Bregman generalizing case, together with the natural "extreme" cases that highlight the importance of which generalized Bregman distance is chosen.

We construct a general framework for deriving error bounds for conic feasibility problems. In particular, our approach allows one to work with cones that fail to be amenable or even to have computable projections, two previously challenging barriers. For the purpose, we first show how error bounds may be constructed using objects called one-step facial residual functions. Then, we develop several tools to compute these facial residual functions even in the absence of closed form expressions for the projections onto the cones. We demonstrate the use and power of our results by computing tight error bounds for the exponential cone feasibility problem. Interestingly, we discover a natural example for which the tightest error bound is related to the Boltzmann-Shannon entropy. We were also able to produce an example of sets for which a H\"{o}lderian error bound holds but the supremum of the set of admissible exponents is not itself an admissible exponent., Comment: 39 pages, comments welcome. In this version, we simplified the error bound framework and also obtained a better error bound result for products of exponential cones

We revisit the feasibility approach to the construction of compactly supported smooth orthogonal wavelets on the line. We highlight its flexibility and illustrate how symmetry and cardinality properties are easily embedded in the design criteria. We solve the resulting wavelet feasibility problems using recently introduced centering methods, and we compare performance. Solutions admit real-valued compactly supported smooth orthogonal scaling functions and wavelets with near symmetry and near cardinality properties.

The problem of minimizing an entropy functional subject to linear constraints is a useful example of partially finite convex programming. In the 1990s, Borwein and Lewis provided broad and easy-to-verify conditions that guarantee strong duality for such problems. Their approach is to construct a function in the quasi-relative interior of the relevant infinite-dimensional set, which assures the existence of a point in the core of the relevant finite-dimensional set. We revisit this problem, and provide an alternative proof by directly appealing to the definition of the core, rather than by relying on any properties of the quasi-relative interior. Our approach admits a minor relaxation of the linear independence requirements in Borwein and Lewis' framework, which allows us to work with certain piecewise-defined moment functions precluded by their conditions. We provide such a computed example that illustrates how this relaxation may be used to tame observed Gibbs phenomenon when the underlying data is discontinuous. The relaxation illustrates the understanding we may gain by tackling partially-finite problems from both the finite-dimensional and infinite-dimensional sides. The comparison of these two approaches is informative, as both proofs are constructive.

For many problems, some of which are reviewed in the paper, popular algorithms like Douglas--Rachford (DR), ADMM, and FISTA produce approximating sequences that show signs of spiraling toward the solution. We present a meta-algorithm that exploits such dynamics to potentially enhance performance. The strategy of this meta-algorithm is to iteratively build and minimize surrogates for the Lyapunov function that captures those dynamics. As a first motivating application, we show that for prototypical feasibility problems the circumcentered-reflection method (CRM), subgradient projections, and Newton--Raphson are all describable as gradient-based methods for minimizing Lyapunov functions constructed for DR operators, with the former returning the minimizers of spherical surrogates for the Lyapunov function. As a second motivating application, we introduce a new method that shares these properties but with the added advantages that it: 1) does not rely on subproblems (e.g. reflections) and so may be applied for any operator whose iterates have the spiraling property; 2) provably has the aforementioned Lyapunov properties with few structural assumptions and so is generically suitable for primal/dual implementation; and 3) maps spaces of reduced dimension into themselves whenever the original operator does. This makes possible the first primal/dual implementation of a method that seeks the center of spiraling iterates. We describe this method, and provide a computed example (basis pursuit).

Recently, circumcentering reflection method (CRM) has been introduced for solving the feasibility problem of finding a point in the intersection of closed constraint sets. It is closely related with Douglas--Rachford method (DR). We prove local convergence of CRM in the same prototypical settings of most theoretical analysis of regular nonconvex DR, whose consideration is made natural by the geometry of the phase retrieval problem. For the purpose, we show that CRM is related to the method of subgradient projections. For many cases when DR is known to converge to a feasible point, we establish that CRM locally provides a better convergence rate. As a root finder, we show that CRM has local convergence whenever Newton--Raphson method does, has quadratic rate whenever Newton--Raphson method does, and exhibits superlinear convergence in many cases when Newton--Raphson method fails to converge at all. We also obtain explicit regions of convergence. As an interesting aside, we demonstrate local convergence of CRM to feasible points in cases when DR converges to fixed points that are not feasible. We demonstrate an extension in higher dimensions, and use it to obtain convergence rate guarantees for sphere and subspace feasibility problems. Armed with these guarantees, we experimentally discover that CRM is highly sensitive to compounding numerical error that may cause it to achieve worse rates than those guaranteed by theory. We then introduce a numerical modification that enables CRM to achieve the theoretically guaranteed rates. Any future works that study CRM for product space formulations of feasibility problems should take note of this sensitivity and account for it in numerical implementations.

Recently, a new distance has been introduced for the graphs of two point-to-set operators, one of which is maximally monotone. When both operators are the subdifferential of a proper lower semicontinuous convex function, this distance specializes under modest assumptions to the classical Bregman distance. We name this new distance the generalized Bregman distance, and we shed light on it with examples that utilize the other two most natural representative functions: the Fitzpatrick function and its conjugate. We provide sufficient conditions for convexity, coercivity, and supercoercivity: properties that are essential for implementation in proximal point type algorithms. We establish these results for both the left and right variants of this new distance. We construct examples closely related to the Kullback--Leibler divergence, which was previously considered in the context of Bregman distances, and whose importance in information theory is well known. In so doing, we demonstrate how to compute a difficult Fitzpatrick conjugate function, and we discover natural occurrences of the Lambert $\W$ function, whose importance in optimization is of growing interest.

Phase plotting is a useful way of visualising functions on complex space. We reinvent the method in the context of hyperbolic geometry, and we use it to plot functions on various representative surfaces for hyperbolic space, illustrating with direct motions in particular. The reinvention is nontrivial, and we discuss the essential features for ensuring that such visualisations convey useful information. Our approach is to exploit conformal maps between representative surfaces, in order to uniquely represent the preimages of geodesics. Our considerations and methods are prototypical of what one might consider for adapting similar methods of visualisation in other contexts.

We focus on the convergence analysis of averaged relaxations of cutters, specifically for variants that---depending upon how parameters are chosen---resemble \emph{alternating projections}, the \emph{Douglas--Rachford method}, \emph{relaxed reflect-reflect}, or the \emph{Peaceman--Rachford} method. Such methods are frequently used to solve convex feasibility problems. The standard convergence analyses of projection algorithms are based on the \emph{firm nonexpansivity} property of the relevant operators. However if the projections onto the constraint sets are replaced by cutters (projections onto separating hyperplanes), the firm nonexpansivity is lost. We provide a proof of convergence for a family of related averaged relaxed cutter methods under reasonable assumptions, relying on a simple geometric argument. This allows us to clarify fine details related to the allowable choice of the relaxation parameters, highlighting the distinction between the exact (firmly nonexpansive) and approximate (strongly quasi-nonexpansive) settings. We provide illustrative examples and discuss practical implementations of the method.

The Douglas--Rachford method is a splitting method frequently employed for finding zeroes of sums of maximally monotone operators. When the operators in question are normal cones operators, the iterated process may be used to solve feasibility problems of the form: Find $x \in \bigcap_{k=1}^N S_k.$ The success of the method in the context of closed, convex, nonempty sets $S_1,\dots,S_N$ is well-known and understood from a theoretical standpoint. However, its performance in the nonconvex context is less understood yet surprisingly impressive. This was particularly compelling to Jonathan M. Borwein who, intrigued by Elser, Rankenburg, and Thibault's success in applying the method for solving Sudoku Puzzles, began an investigation of his own. We survey the current body of literature on the subject, and we summarize its history. We especially commemorate Professor Borwein's celebrated contributions to the area.

In their 2016 article `Meetings with Lambert $\mathcal{W}$ and Other Special Functions in Optimization and Analysis', Borwein and Lindstrom considered the minimization of an entropy functional utilizing the weighted average of the negative Boltzmann--Shannon entropy $x\mapsto x\log x -x$ and the energy $x \mapsto x^2/2$; the solution employed the Fenchel-Moreau conjugate of the sum. However, in place of a traditional arithmetic average of two convex functions, it is also, perhaps even more, natural to use their proximal average. We explain the advantages and illustrate them by computing the analogous proximal averages for the negative entropy and energy. We use the proximal averages to solve entropy functional minimization problems similar to those considered by Borwein and Lindstrom, illustrating the benefits of using a true homotopy. Through experimentation, we discover computational barriers to obtaining solutions when computing with proximal averages, and we demonstrate a method which appears to remedy them.

The Douglas--Rachford algorithm is a popular algorithm for solving both convex and nonconvex feasibility problems. While its behaviour is settled in the convex inconsistent case, the general nonconvex inconsistent case is far from being fully understood. In this paper, we focus on the most simple nonconvex inconsistent case: when one set is a hyperplane and the other a doubleton (i.e., a two-point set). We present a characterization of cycling in this case which --- somewhat surprisingly --- depends on whether the ratio of the distance of the points to the hyperplane is rational or not. Furthermore, we provide closed-form expressions as well as several concrete examples which illustrate the dynamical richness of this algorithm.

The Douglas-Rachford method has been employed successfully to solve many kinds of non-convex feasibility problems. In particular, recent research has shown surprising stability for the method when it is applied to finding the intersections of hypersurfaces. Motivated by these discoveries, we reformulate a second order boundary valued problem (BVP) as a feasibility problem where the sets are hypersurfaces. We show that such a problem may always be reformulated as a feasibility problem on no more than three sets and is well-suited to parallelization. We explore the stability of the method by applying it to several examples of BVPs, including cases where the traditional Newton's method fails.

Moreau's seminal paper, introducing what is now called the Moreau envelope and the proximity operator (also known as the proximal mapping), appeared in 1965. The Moreau envelope of a given convex function provides a regularized version which has additional desirable properties such as differentiability and full domain. Fifty years ago, Attouch proposed using the Moreau envelope for regularization. Since then, this branch of convex analysis has developed in many fruitful directions. In 1967, Bregman introduced what is nowadays known as the Bregman distance as a measure of discrepancy between two points generalizing the square of the Euclidean distance. Proximity operators based on the Bregman distance have become a topic of significant research as they are useful in the algorithmic solution of optimization problems. More recently, in 2012, Kan and Song studied regularization aspects of the left Bregman-Moreau envelope even for nonconvex functions. In this paper, we complement previous works by analyzing the left and right Bregman-Moreau envelopes and by providing additional asymptotic results. Several examples are provided.

We expand upon previous work that examined behavior of the iterated Douglas-Rachford method for a line and a circle by considering two generalizations: that of a line and an ellipse and that of a line together with a $p$-sphere. With computer assistance we discover a beautiful geometry that illustrates phenomena which may affect the behavior of the iterates by slowing or inhibiting convergence for feasible cases. We prove local convergence near feasible points, and---seeking a better understanding of the behavior---we employ parallelization in order to study behavior graphically. Motivated by the computer-assisted discoveries, we prove a result about behavior of the method in infeasible cases.