Search

Weight clipping is a well-known strategy to keep the Lipschitz constant of the critic under control, in Wasserstein GAN training. After each training iteration, all parameters of the critic are clipped to a given box, impacting the progress made by the optimizer. In this work, we propose a new strategy for weight clipping in Wasserstein GANs. Instead of directly clipping the parameters, we first obtain an equivalent model that is closer to the clipping box, and only then clip the parameters. Our motivation is to decrease the impact of the clipping strategy on the objective, at each iteration. This equivalent model is obtained by following invariant curves in the critic loss landscape, whose existence is a consequence of the positive homogeneity of common activations: rescaling the input and output signals to each activation by inverse factors preserves the loss. We provide preliminary experiments showing that the proposed strategy speeds up training on Wasserstein GANs with simple feed-forward architectures.

Orthogonal regularizers typically promote column orthonormality of some matrix W ∈ ℝ n×p , by measuring the discrepancy between W ⊤ W and the identity according to some matrix norm. This paper explores the behavior of these regularizers when W is horizontal (n < p), so that column orthonormality cannot be achieved. Our motivation comes from orthogonal regularization of feed-forward neural networks: it is there desired to regularize all (vertical and horizontal) weight matrices of the model.One possible solution to address this issue is to transpose horizontal matrices before regularization. We prove that transposition is useless for the Frobenius norm (squared), as the corresponding regularizer promotes simultaneously orthonormality of the rows and of the columns of W. On the other hand, we highlight important qualitative differences with newer regularizers, including the MC and SRIP orthogonal regularizers. We conclude the paper with some numerical results supporting our theoretical findings.

We propose a differential geometric approach for building families of low-rank covariance matrices, via interpolation on low-rank matrix manifolds. In contrast with standard parametric covariance classes, these families offer significant flexibility for problem-specific tailoring via the choice of “anchor” matrices for interpolation, for instance over a grid of relevant conditions describing the underlying stochastic process. The interpolation is computationally tractable in high dimensions, as it only involves manipulations of low-rank matrix factors. We also consider the problem of covariance identification, i.e., selecting the most representative member of the covariance family given a data set. In this setting, standard procedures such as maximum likelihood estimation are nontrivial because the covariance family is rank-deficient; we resolve this issue by casting the identification problem as distance minimization. We demonstrate the utility of these differential geometric families for interpolation and identification in a practical application: wind field covariance approximation for unmanned aerial vehicle navigation.

We propose a differential geometric construction for families of low-rank covariance matrices, via interpolation on low-rank matrix manifolds. In contrast with standard parametric covariance classes, these families offer significant flexibility for problem-specific tailoring via the choice of “anchor” matrices for the interpolation. Moreover, their low-rank facilitates computational tractability in high dimensions and with limited data. We employ these covariance families for both interpolation and identification, where the latter problem comprises selecting the most representative member of the covariance family given a data set. In this setting, standard procedures such as maximum likelihood estimation are nontrivial because the covariance family is rank-deficient; we resolve this issue by casting the identification problem as distance minimization. We demonstrate the power of these differential geometric families for interpolation and identification in a practical application: wind field covariance approximation for unmanned aerial vehicle navigation.

We consider the manifold of rank-p positive-semidefinite matrices of size n, seen as a quotient of the set of full-rank n-by-p matrices by the orthogonal group in dimension p. The resulting distance coincides with the Wasserstein distance between centered degenerate Gaussian distributions. We obtain expressions for the Riemannian curvature tensor and the sectional curvature of the manifold. We also provide tangent vectors spanning planes associated with the extreme values of the sectional curvature.

—We consider model order reduction of linear timevarying systems on a finite time interval using balanced truncation. A standard way to perform MOR is to first numerically integrate the associated pair of differential Lyapunov equations for the two gramians, then compute projection matrices using the square root method, and finally formulate the reduced systems at each time instant of a chosen grid. This approach is well-knownfordeliveringgoodapproximation,butrathercostly in computation and storage requirement. Furthermore, if one needs to compute the reduced system for any new time instant thatisnotincludedinthechosengrid,thementionedprocedure must be performed again without explicitly making use of the already computed data. For dealing with such a situation, we propose to store the projection matrices corresponding to a simplified sparse time grid and to use them to recover the projection subspaces at any other time instant via curve interpolation on the Grassmann manifold. By doing this, we can avoid the repetition of solving the differential Lyapunov equations which is the most expensive step in the procedure and therefore, as shown in a numerical example, accelerate the online reduction process.

We present several interpolation schemes on the manifold of fixed-rank positive-semidefinite (PSD) matrices. We explain how these techniques can be used for model order reduction of parameterized linear dynamical systems, and obtain preliminary results on an application.

This thesis contributes to the emerging topic of data fitting algorithms that rely on the structure of the data. It tackles two problems: averaging a set of positive-definite (PD) matrices, and fitting a curve to a set of positive-semidefinite (PSD) matrices. Averaging a set of PD matrices is a subtask in several medical imaging and image processing algorithms. It is also used in classifiers for electroencephalogram signals in brain-computer interfaces. This last application motivates the development of fast and incremental averaging tools on PD matrices, to allow online learning and adaptation of the classifier. We propose and study accelerated versions of two algorithms for averaging PD matrices. The first is an incremental gradient descent algorithm on the manifold. The second computes the mean of the data points in a decentralized way, by relying on the expression for the mean of two data points. Finally, we apply our incremental algorithm to the EEG classification problem. For the second task, we restrict ourselves to the set of fixed-rank PSD matrices, to take advantage of the differentiable structure of this set. This restriction is applicable, for example, when working with low-rank estimations of large PSD matrices, or with large covariance matrices characterizing a dynamical process of lower dimension. We compute expressions for the main tools required for applying curve fitting algorithms to the manifold of fixed-rank PSD matrices. We finally apply these algorithms to two problems: parametric model order reduction and wind field estimation., (FSA - Sciences de l'ingénieur) -- UCL, 2019

n this paper, we tackle the problem of action recognition using body skeletons extracted from video sequences. Our approach lies in the continuity of recent works representing video frames by Gramian matrices that describe a trajectory on the Riemannian manifold of positive-semidefinite matrices of fixed rank. Compared to previous work, the manifold of fixed-rank positive-semidefinite matrices is endowed with a different metric, and we resort to different algorithms for the curve fitting and temporal alignment steps. We evaluated our approach on three publicly available datasets (UTKinect-Action3D, KTH-Action and UAVGesture). The results of the proposed approach are competitive with respect to state-of-the-art methods, while only involving body skeletons.

We present a method to compute a fitting curve B to a set of data points d0,...,dm lying on a manifold M. That curve is obtained by blending together Euclidean Bézier curves obtained on different tangent spaces. The method guarantees several properties among which B is C1 and is the natural cubic smoothing spline when M is the Euclidean space. We show examples on the sphere S2 as a proof of concept.

We propose several methods that address the problem of fitting a $C^1$ curve $\gamma$ to time-labeled data points on a manifold. The methods have a parameter, $\lambda$, to adjust the relative importance of the two goals that the curve should meet: being “straight enough” while fitting the data “closely enough.” The methods are designed for ease of use: they only require to compute Riemannian exponentials and logarithms, they represent the curve by means of a number of tangent vectors that grows linearly with the number of data points, and, once the representation is computed, evaluating $\gamma(t)$ at any $t$ requires a small number of exponentials and logarithms (independent of the number of data points). Among the proposed methods, the blended cubic spline technique combines the additional properties of interpolating the data when $\lambda \rightarrow \infty$ and reducing to the well-known cubic smoothing spline when the manifold is Euclidean. The methods are illustrated on synthetic and real data.

The translation of brain activity into user command, through Brain-Computer Interfaces (BCI), is a very active topic in machine learning and signal processing. As commercial applications and out-of-the-lab solutions are proposed, there is an increased pressure to provide online algorithms and real-time implementations. Electroencephalography (EEG) systems offer lightweight and wearable solutions, at the expense of signal quality. Approaches based on covariance matrices have demonstrated good robustness to noise and provide a suitable representation for classification tasks, relying on advances in Riemannian geometry. We propose to equip the minimum distance to mean (MDM) classifier with a new family of means, based on the inductive mean, for block-online classification tasks and to embed the inductive mean in an incremental learning algorithm for online classification of EEG.

We consider the problem of extending any two-variable mean M(·, ·) to a multi-variable mean, using no other tool than M(·, ·) itself. Pálfia proposed an iterative procedure that consists in evaluating successively two-variable means according to a cyclic pattern. We propose here a variant of his procedure to improve the convergence speed. Our approach consists in re-ordering the iterates after each iteration in order to speed up the transfer of information between successive iterates.