Search

We review the basic outline of the highly successful diffusion Monte Carlo technique commonly used in contexts ranging from electronic structure calculations to rare event simulation and data assimilation, and propose a new class of randomized iterative algorithms based on similar principles to address a variety of common tasks in numerical linear algebra. From the point of view of numerical linear algebra, the main novelty of the Fast Randomized Iteration schemes described in this article is that they work in either linear or constant cost per iteration (and in total, under appropriate conditions) and are rather versatile: we will show how they apply to solution of linear systems, eigenvalue problems, and matrix exponentiation, in dimensions far beyond the present limits of numerical linear algebra. While traditional iterative methods in numerical linear algebra were created in part to deal with instances where a matrix (of size $\mathcal{O}(n^2)$) is too big to store, the algorithms that we propose are effective even in instances where the solution vector itself (of size $\mathcal{O}(n)$) may be too big to store or manipulate. In fact, our work is motivated by recent DMC based quantum Monte Carlo schemes that have been applied to matrices as large as $10^{108} \times 10^{108}$. We provide basic convergence results, discuss the dependence of these results on the dimension of the system, and demonstrate dramatic cost savings on a range of test problems., 44 pages, 7 figures

Matrix computations lie at the heart of many scientific computations. While sophisticated algorithms have been established for various numerical linear algebra problems such as the solution of linear simultaneous equations and eigenvalue problems, they require considerable modification with the advent of exaFLOPS- scale supercomputers, which are expected to have a huge number of computing cores, deep memory hierarchy, and increased probability of hardware errors. In this chapter, we discuss projected hardware characteristics of exaFLOPS machines and summarize the challenges to be faced by numerical linear algebra algorithms in the near future. Based on these preparations, we present a brief survey of recent research efforts in the field of numerical linear algebra targeted at meeting these challenges.

This survey provides an introduction to the use of randomization in the design of fast algorithms for numerical linear algebra. These algorithms typically examine only a subset of the input to solve basic problems approximately, including matrix multiplication, regression and low-rank approximation. The survey describes the key ideas and gives complete proofs of the main results in the field. A central unifying idea is sampling the columns (or rows) of a matrix according to their squared lengths.

Les méthodes aléatoires pour le calcul approché de décomposition aux valeur singulières/valeurs propres ont suscité beaucoup d’intérêt au cours des dernières décennies. Ces méthodes se sont avérées performantes, efficaces en termes de coût de calcul et particulièrement bien adaptées aux problèmes de grande taille. À cet égard, des recherches récentes ont proposé des applications de cesméthodes en assimilation de données, où la taille des problèmes est prohibitive pour un grand nombre d'approches classiques. Dans cette thèse, nous proposons trois contributions interconnectées aux méthodes aléatoires pour l'approximation de rang faible, l'extraction d’information spectrale et le préconditionnement en assimilation de données variationnelle.Premièrement, nous proposons une analyse générale de l'erreur d'approximation de rang faible aléatoire en norme de Frobenius et en norme spectrale. Cette généralisation étend les possibilités d'analyse à un plus grand nombre de méthodes aléatoires en autorisant des matrices de covariance générales et un vecteur de moyenne non nulle pour la matrice gaussienne d'échantillonnage. La particularisation de nos bornes à la méthode dite de Randomized Singular Value Decomposition (RSVD) montre que nous améliorons les bornes d'erreur de référence proposées par Halko, Martinsson et Tropp (2011).Ensuite, nous présentons des algorithmes aléatoires pour la résolution de problèmes aux valeurs propres spécifiques qui apparaissent notamment en assimilation de données. Les méthodes proposées sont polyvalentes et généralisent les contributions de Saibaba, Lee et Kitanidis (2016) et Daužickaité et al. (2021). Nous fournissons ensuite une analyse théorique de nos méthodes qui éclaire sur la sensibilité de l’erreur au nombre d'itérations de sous-espace, au nombre d'échantillons aléatoires et à la matrice de covariance pour la matrice gaussienne d'échantillonnage. Des illustrations numériques sur un problème d'assimilation de données confirment le potentiel de nos algorithmes.Enfin, nous proposons une classe de préconditionnement à mémoire limitée aléatoire dédiée à l'assimilation de données variationnelle. Nous proposons de tels préconditionnements pour deux méthodes de Krylov en particulier: une approche dite inverse-free dans l'espace primal introduite par Guröl (2013) et une méthode d'espace dual proposée par Gratton et Tshimanga (2009). La dimension réduite de l'espace dual rend cette dernière approche plus intéressante à la fois en termes de coût de calcul et de stockage. Les préconditionnements aléatoires proposés sont basés sur des expressions adaptées identifiées par Gürol (2013) pour lesquelles les calculs coûteux d’information spectrale exacte est remplacée par des approximations obtenues avec une procédure aléatoire. Des illustrations sur un problème d'assimilation de données variationnel quadridimensionnel de référence démontrent le potentiel de nos préconditionnements aléatoires, ouvrant ainsi des perspectives intéressantes. Randomized methods for computing approximate singular value/eigenvalue decompositions have gained a lot of attention during the past decades. These methods have proven to perform well, are computationally efficient, and are well suited for large scale applications. In this regard, recent researches have proposed successful applications of randomized algorithms in data assimilation, where the huge size of the problems is prohibitive for a large number of standard approaches. In this thesis, we propose three interconnected contributions in randomized methods for low-rank approximation, extraction of eigenpairs, and preconditioning within variational data assimilation.First, we propose a general error analysis of randomized low-rank approximation in Frobenius and spectral norms. This generalization extends the possibilities of analysis to a larger class of randomized methods by allowing general covariance matrices and non-zero mean for the Gaussian sample matrix. Particularization of our bounds to the Randomized Singular Value Decomposition (RSVD) shows that we improve the reference error bounds due to Halko, Martinsson and Tropp (2011).Then, we develop randomized algorithms to address specific eigenvalue problems that naturally arise in data assimilation. The proposed methods are versatile, and generalize the contributions from Saibaba, Lee and Kitanidis (2016) and Daužickaitė et al. (2021). We then provide a theoretical analysis of the methods, which gives insights regarding the number of subspace iterations, number of random samples, and optimal covariance matrix of the Gaussian sample matrix. Numerical illustrations on a data assimilation test problem confirm the potential of our algorithms.Finally, we propose a class of randomized spectral limited memory preconditioners for variational data assimilation. We provide such preconditioners for two given Krylov subspace methods: an inverse-free approach in the primal space introduced by Guröl (2013) and a dual space method proposed by Gratton and Tshimanga (2009). The reduced dimension of the dual space makes this latter approach computationally efficient both in terms of cost and storage.Our randomized spectral limited memory preconditioners are based on appropriate expressions identified by Gürol (2013) where we replace expensive computations of exact eigenpairs by approximations obtained with a randomized procedure. Illustrations on a benchmark four-dimensional variational data assimilation problem prove that our randomized preconditioners perform well, opening interesting perspectives.

My dissertation project addresses the integration of numerical linear algebra approaches to the field of computer graphics and visualization, especially physically-based Fluid Animations. I mainly focus on (matrix) decomposition techniques with regard to spectral theory and related concepts. In many applications of Fluid Animations, in particular particle-based ones, these techniques are not consulted due to its irregular nature. However, the use of spectral-theoretic approaches leads to both theoretical insights and practical enhancements of the algorithm in terms of precision, efficiency, and stability. My research is conducted at the University of Stuttgart and at the Hochschule der Medien, Stuttgart, where I am a member of the joint graduate school Digital Media.

Today’s floating-point arithmetic landscape is broader than ever. While scientific computing has traditionally used single precision and double precision floating-point arithmetics, half precision is increasingly available in hardware and quadruple precision is supported in software. Lower precision arithmetic brings increased speed and reduced communication and energy costs, but it produces results of correspondingly low accuracy. Higher precisions are more expensive but can potentially provide great benefits, even if used sparingly. A variety of mixed precision algorithms have been developed that combine the superior performance of lower precisions with the better accuracy of higher precisions. Some of these algorithms aim to provide results of the same quality as algorithms running in a fixed precision but at a much lower cost; others use a little higher precision to improve the accuracy of an algorithm. This survey treats a broad range of mixed precision algorithms in numerical linear algebra, both direct and iterative, for problems including matrix multiplication, matrix factorization, linear systems, least squares, eigenvalue decomposition and singular value decomposition. We identify key algorithmic ideas, such as iterative refinement, adapting the precision to the data, and exploiting mixed precision block fused multiply–add operations. We also describe the possible performance benefits and explain what is known about the numerical stability of the algorithms. This survey should be useful to a wide community of researchers and practitioners who wish to develop or benefit from mixed precision numerical linear algebra algorithms.