Search

Finding zeros of the sum of two maximally monotoneoperators involving a continuous linear operator is a central problem in optimization and monotone operator theory. We revisit the duality framework proposed by Eckstein, Ferris, Pennanen, and Robinson from a quarter of a century ago. Paramonotonicity is identified as a broad condition ensuring that saddle points coincide with the closed convex rectangle formed by the primal and dual solutions. Additionally, we characterize total duality in the subdifferential setting and derive projection formulas for sets that arise in the analysis of the Chambolle-Pock algorithm within the recent framework developed by Bredies, Chenchene, Lorenz, and Naldi.

It is well known that the iterates of an averaged nonexpansive mapping may only converge weakly to fixed point. A celebrated result by Baillon, Bruck, and Reich from 1978 yields strong convergence in the presence of linearity. In this paper, we extend this result to allow for flexible relaxation parameters. Examples are also provided to illustrate the results.

Monotone inclusion problems occur in many areas of optimization and variational analysis. Splitting methods, which utilize resolvents or proximal mappings of the underlying operators, are often applied to solve these problems. In 2022, Bredies, Chenchene, Lorenz, and Naldi introduced a new elegant algorithmic framework that encompasses various well known algorithms including Douglas-Rachford and Chambolle-Pock. They obtained powerful weak and strong convergence results, where the latter type relies on additional strong monotonicity assumptions. In this paper, we complement the analysis by Bredies et al. by relating the projections of the fixed point sets of the underlying operators that generate the (reduced and original) preconditioned proximal point sequences. We also obtain strong convergence results in the case of linear relations. Various examples are provided to illustrate the applicability of our results.

Projection operators are fundamental algorithmic operators in Analysis and Optimization. It is well known that these operators are firmly nonexpansive; however, their composition is generally only averaged and no longer firmly nonexpansive. In this note, we introduce the modulus of averagedness and provide an exact result for the composition of two linear projection operators. As a consequence, we deduce that the Ogura-Yamada bound for the modulus of the composition is sharp.

The solution of the cubic equation has a century-long history; however, the usual presentation is geared towards applications in algebra and is somewhat inconvenient to use in optimization where frequently the main interest lies in real roots. In this note, we present the roots of the cubic in a form that makes them convenient to use and we also focus on information on the location of the real roots. Armed with this, we provide several applications in optimization where we compute Fenchel conjugates, proximal mappings and projections., Comment: 26 pages, 5 figures

The Fenchel-Young inequality is fundamental in Convex Analysis and Optimization. It states that the difference between certain function values of two vectors and their inner product is nonnegative. Recently, Carlier introduced a very nice sharpening of this inequality, providing a lower bound that depends on a positive parameter. In this note, we expand on Carlier's inequality in three ways. First, a duality statement is provided. Secondly, we discuss asymptotic behaviour as the underlying parameter approaches zero or infinity. Thirdly, relying on cyclic monotonicity and associated Fitzpatrick functions, we present a lower bound that features an infinite series of squares of norms. Several examples illustrate our results.

Let $A$ be a closed affine subspace and let $B$ be a hyperplane in a Hilbert space. Suppose we are given their associated nearest point mappings $P_A$ and $P_B$, respectively. We present a formula for the projection onto their intersection $A\cap B$. As a special case, we derive a formula for the projection onto the intersection of two hyperplanes. These formulas provides useful information even if $A\cap B$ is empty. Examples and numerical experiments are also provided.

In $\mathbb{R}^3$, a hyperbolic paraboloid is a classical saddle-shaped quadric surface. Recently, Elser has modeled problems arising in Deep Learning using rectangular hyperbolic paraboloids in $\mathbb{R}^n$. Motivated by his work, we provide a rigorous analysis of the associated projection. In some cases, finding this projection amounts to finding a certain root of a quintic or cubic polynomial. We also observe when the projection is not a singleton and point out connections to graphical and set convergence., Comment: 22 pages, 1 figure, fixed references in latest version

Let $C$ be a closed convex subset of a real Hilbert space containing the origin, and assume that $K$ is the homogenization cone of $C$, i.e., the smallest closed convex cone containing $C \times \{1\}$. Homogenization cones play an important role in optimization as they include, for instance, the second-order/Lorentz/"ice cream" cone. In this note, we discuss the polar cone of $K$ as well as an algorithm for finding the projection onto $K$ provided that the projection onto $C$ is available. Various examples illustrate our results., Comment: Twenty-seven pages, four figures

Finding a zero of a sum of maximally monotone operators is a fundamental problem in modern optimization and nonsmooth analysis. Assuming that the resolvents of the operators are available, this problem can be tackled with the Douglas-Rachford algorithm. However, when dealing with three or more operators, one must work in a product space with as many factors as there are operators. In groundbreaking recent work by Ryu and by Malitsky and Tam, it was shown that the number of factors can be reduced by one. A similar reduction was achieved recently by Campoy through a clever reformulation originally proposed by Kruger. All three splitting methods guarantee weak convergence to some solution of the underlying sum problem; strong convergence holds in the presence of uniform monotonicity. In this paper, we provide a case study when the operators involved are normal cone operators of subspaces and the solution set is thus the intersection of the subspaces. Even though these operators lack strict convexity, we show that striking conclusions are available in this case: strong (instead of weak) convergence and the solution obtained is (not arbitrary but) the projection onto the intersection. Numerical experiments to illustrate our results are also provided., Comment: 25 Pages, 3 Figures. arXiv admin note: substantial text overlap with arXiv:2109.11072; final version that was submitted to SIOpt

Recently, Simons provided a lemma for a support function of a closed convex set in a general Hilbert space and used it to prove the geometry conjecture on cycles of projections. In this paper, we extend Simons's lemma to closed convex functions, show its connections to Attouch-Thera duality, and use it to characterize (classical and phantom) cycles and gap vectors of proximal mappings.

Sets of bilinear constraints are important in various machine learning models. Mathematically, they are hyperbolas in a product space. In this paper, we give a complete formula for projections onto sets of bilinear constraints or hyperbolas in a general Hilbert space., Comment: 16 pages

Finding a zero of a sum of maximally monotone operators is a fundamental problem in modern optimization and nonsmooth analysis. Assuming that resolvents of the operators are available, this problem can be tackled with the Douglas-Rachford algorithm. However, when dealing with three or more operators, one must work in a product space with as many factors as there are operators. In groundbreaking recent work by Ryu and by Malitsky and Tam, it was shown that the number of factors can be reduced by one. These splitting methods guarantee weak convergence to some solution of the underlying sum problem; strong convergence holds in the presence of uniform monotonicity. In this paper, we provide a case study when the operators involved are normal cone operators of subspaces and the solution set is thus the intersection of the subspaces. Even though these operators lack strict convexity, we show that striking conclusions are available in this case: strong (instead of weak) convergence and the solution obtained is (not arbitrary but) the projection onto the intersection. Numerical experiments to illustrate our results are also provided, Comment: 25 pages, 4 figures

We provide a proximal average with repect to a $1$-coercive Legendre function. In the sense of Bregman distance, the Bregman envelope of the proximal average is a convex combination of Bregman envelopes of individual functions. The Bregman proximal mapping of the average is a convex combination of convexified proximal mappings of individual functions. Techniques from variational analysis provide the keys for the Bregman proximal average.

We consider the set of pairs of orthogonal vectors in Hilbert space, which is also called the cross because it is the union of the horizontal and vertical axes in the Euclidean plane when the underlying space is the real line. Crosses, which are nonconvex sets, play a significant role in various branches of nonsmooth analysis such as feasibility problems and optimization problems. In this work, we study crosses and show that in infinite-dimensional settings, they are never weakly (sequentially) closed. Nonetheless, crosses do turn out to be proximinal (i.e., they always admit projections) and we provide explicit formulas for the projection onto the cross in all cases.

Predictive computational modeling of polymer materials is necessary for the efficient design of composite materials and the corresponding processing methods. Molecular dynamics (MD) modeling is especially important for establishing accurate processing-structure-property relationships for neat resins. For MD modeling of amorphous polymer materials, an accurate force field is fundamental to reliable prediction of properties. Reactive force fields, in which chemical bonds can be formed or broken, offer further capability in predicting the mechanical behavior of amorphous polymers subjected to relatively large deformations. To this end, the Reactive Interface Force Field (IFF-R) has been recently developed to provide an efficient means to predict the behavior of materials under these conditions. Although IFF-R has been proven to be accurate for some crystalline organic and inorganic systems, it has not yet been proven to be accurate for amorphous polymer systems. The objective of this study is to use IFF-R to predict the thermo-mechanical properties of three different epoxy systems and validate with experimental measurements. The results indicate that IFF-R predicts thermo-mechanical properties that agree closely with experiment. Therefore, IFF-R can be used to reliably establish mechanical properties of polymers on the molecular level for future design of new composite materials and processing methods.

The notion of Fej\'er monotonicity is instrumental in unifying the convergence proofs of many iterative methods, such as the Krasnoselskii-Mann iteration, the proximal point method, the Douglas-Rachford splitting algorithm, and many others. In this paper, we present directionally asymptotical results of strongly convergent subsequences of Fej\'er monotone sequences. We also provide examples to show that the sets of directionally asymptotic cluster points can be large and that weak convergence is needed in infinite-dimensional spaces.

More than 40 years ago, Lions and Mercier introduced in a seminal paper the Douglas-Rachford algorithm. Today, this method is well recognized as a classical and highly successful splitting method to find minimizers of the sum of two (not necessarily smooth) convex functions. While the underlying theory has matured, one case remains a mystery: the behaviour of the shadow sequence when the given functions have disjoint domains. Building on previous work, we establish for the first time weak and value convergence of the shadow sequence generated by the Douglas-Rachford algorithm in a setting of unprecedented generality. The weak limit point is shown to solve the associated normal problem which is a minimal perturbation of the original optimization problem. We also present new results on the geometry of the minimal displacement vector.

In 1990, Romero presented a beautiful formula for the projection onto the set of rectangular matrices with prescribed row and column sums. Variants of Romero's formula have been rediscovered by Khoury and by Glunt, Hayden, and Reams, for bistochastic (square) matrices in 1998. These results have found various generalizations and applications. In this paper, we provide a formula for the more general problem of finding the projection onto the set of rectangular matrices with prescribed scaled row and column sums. Our approach is based on computing the Moore-Penrose inverse of a certain linear operator associated with the problem. In fact, our analysis holds even for Hilbert-Schmidt operators and we do not have to assume consistency. We also perform numerical experiments featuring the new projection operator.

We compute the minimal angle spread with respect to the uniform distribution in the probability simplex. The resulting optimization problem is analytically solved. The formula provided shows that the minimal angle spread approaches zero as the dimension tends to infinity. We also discuss an application in cognitive science.

There are two basic angles associated with a pair of linear subspaces: the Diximier angle and the Friedrichs angle. The Dixmier angle of the pair of orthogonal complements is the same as the Dixmier angle of the original pair provided that the original pair gives rise to a direct (not necessarily orthogonal) sum of the underlying Hilbert space. The Friedrichs angles of the original pair and the pair of the orthogonal complements always coincide. These two results are due to Krein, Krasnoselskii, and Milman and to Solmon, respectively. In 1995, Deutsch provided a very nice survey with complete proofs and interesting historical comments. One key result in Deutsch's survey was an inequality for Dixmier angles provided by Hundal. In this paper, we present extensions of these results to the case when the linear subspaces are only required to be convex cones. It turns out that Hundal's result has a nice conical extension while the situation is more technical for the results by Krein et al.\ and by Solmon. Our analysis is based on Deutsch's survey and our recent work on angles between convex sets. Throughout, we also provide examples illustrating the sharpness of our results., Comment: arXiv admin note: text overlap with arXiv:2008.09313

The problem of finding a best approximation pair of two sets, which in turn generalizes the well known convex feasibility problem, has a long history that dates back to work by Cheney and Goldstein in 1959. In 2018, Aharoni, Censor, and Jiang revisited this problem and proposed an algorithm that can be used when the two sets are finite intersections of halfspaces. Motivated by their work, we present alternative algorithms that utilize projection and proximity operators. Numerical experiments indicate that these methods are competitive and sometimes superior to the one proposed by Aharoni et al.

Using the Attouch-Th\'era duality, we study the cycles, gap vectors and fixed point sets of compositions of proximal mappings. Sufficient conditions are given for the existence of cycles and gap vectors. A primal-dual framework provides an exact relationship between the cycles and gap vectors. We also introduce the generalized cycle and gap vectors to tackle the case when the classical ones do not exist. Examples are given to illustrate our results., Comment: 25 pages

The geometry conjecture, which was posed nearly a quarter of a century ago, states that the fixed point set of the composition of projectors onto nonempty closed convex sets in Hilbert space is actually equal to the intersection of certain translations of the underlying sets. In this paper, we provide a complete resolution of the geometry conjecture. Our proof relies on monotone operator theory. We revisit previously known results and provide various illustrative examples. Comments on the numerical computation of the quantities involved are also presented.

In 1964, Michael Edelstein presented an amazing affine isometry acting on the space of square-summable sequences. This operator has no fixed points, but a suborbit that converges to 0 while another escapes in norm to infinity! We revisit, extend and sharpen his construction. Moreover, we sketch a connection to modern optimization and monotone operator theory.

The notion of the angle between two subspaces has a long history, dating back to Friedrichs's work in 1937 and Dixmier's work on the minimal angle in 1949. In 2006, Deutsch and Hundal studied extensions to convex sets in order to analyze convergence rates for the cyclic projections algorithm. In this work, we characterize the positivity of the minimal angle between two convex cones. We show the existence of, and necessary conditions for, optimal solutions of minimal angle problems associated with two convex subsets as well. Moreover, we generalize a result by Deutsch on minimal angles from linear subspaces to cones. This generalization yields sufficient conditions for the closedness of the sum of two closed convex cones. This also relates to conditions proposed by Beutner and by Seeger and Sossa. Furthermore, we investigate the relation between the intersection of two cones (at least one of which is nonlinear) and the intersection of the polar and dual cones of the underlying cones. It turns out that the two angles involved cannot be positive simultaneously. Various examples illustrate the sharpness of our results., Comment: 18 pages

Maximally monotone operators are fundamental objects in modern optimization. The main classes of monotone operators are subdifferential operators and matrices with a positive semidefinite symmetric part. In this paper, we study a nice class of monotone operators: displacement mappings of isometries of finite order. We derive explicit formulas for resolvents, Yosida approximations, and (set-valued and MoorePenrose) inverses. We illustrate our results by considering certain rational rotators and circular shift operators.

The notion of best approximation mapping (BAM) with respect to a closed affine subspace in finite-dimensional space was introduced by Behling, Bello Cruz and Santos to show the linear convergence of the block-wise circumcentered-reflection method. The best approximation mapping possesses two critical properties of the circumcenter mapping for linear convergence. Because the iteration sequence of BAM linearly converges, the BAM is interesting in its own right. In this paper, we naturally extend the definition of BAM from closed affine subspace to nonempty closed convex set and from $\mathbb{R}^{n}$ to general Hilbert space. We discover that the convex set associated with the BAM must be the fixed point set of the BAM. Hence, the iteration sequence generated by a BAM linearly converges to the nearest fixed point of the BAM. Connections between BAMs and other mappings generating convergent iteration sequences are considered. Behling et al.\ proved that the finite composition of BAMs associated with closed affine subspaces is still a BAM in $\mathbb{R}^{n}$. We generalize their result from $\mathbb{R}^{n}$ to general Hilbert space and also construct a new constant associated with the composition of BAMs. This provides a new proof of the linear convergence of the method of alternating projections. Moreover, compositions of BAMs associated with general convex sets are investigated. In addition, we show that convex combinations of BAMs associated with affine subspaces are BAMs. Last but not least, we connect BAM with circumcenter mapping in Hilbert spaces., Comment: 31 pages and 2 figures

Nonexpansive mappings play a central role in modern optimization and monotone operator theory because their fixed points can describe solutions to optimization or critical point problems. It is known that when the mappings are sufficiently "nice", then the fixed point set of the composition coincides with the intersection of the individual fixed point sets. In this paper, we explore the situation for compositions of linear reflectors. We provide positive results, upper bounds, and limiting examples. We also discuss classical reflectors in the Euclidean plane.

A popular method for finding the projection onto the intersection of two closed convex subsets in Hilbert space is Dykstra's algorithm. In this paper, we provide sufficient conditions for Dykstra's algorithm to converge rapidly, in finitely many steps. We also analyze the behaviour of Dykstra's algorithm applied to a line and a square. This case study reveals stark similarities to the method of alternating projections. Moreover, we show that Dykstra's algorithm may stall for an arbitrarily long time. Finally, we present some open problems.

The circumcentered Douglas--Rachford method (C--DRM), introduced by Behling, Bello Cruz and Santos, is an acceleration of the well-known Douglas-Rachford method (DRM) for finding the best approximation onto the intersection of finitely many affine subspaces. Inspired by the C--DRM, we introduced the more flexible circumcentered reflection method (CRM) and circumcentered isometry method (CIM). The CIM essentially chooses the closest point to the solution among all of the points in an associated affine hull as its iterate and is a generalization of the CRM. The circumcentered--reflection method introduced by Behling et al. to generalize the C--DRM is a special class of our CRM. We consider the CIM induced by a set of finitely many isometries for finding the best approximation onto the intersection of fixed point sets of the isometries which turns out to be an intersection of finitely many affine subspaces. We extend our previous linear convergence results on CRMs in finite-dimensional spaces from reflections to isometries. In order to better accelerate the symmetric method of alternating projections (MAP), the accelerated symmetric MAP first applies another operator to the initial point. (Similarly, to accelerate the DRM, the C--DRM first applies another operator to the initial point as well.) Motivated by these facts, we show results on the linear convergence of CIMs in Hilbert spaces with first applying another operator to the initial point. In particular, under some restrictions, our results imply that some CRMs attain the known linear convergence rate of the accelerated symmetric MAP in Hilbert spaces. We also exhibit a class of CRMs converging to the best approximation in Hilbert spaces with a convergence rate no worse than the sharp convergence rate of MAP. The fact that some CRMs attain the linear convergence rate of MAP or accelerated symmetric MAP is entirely new., Comment: 29 pages