1. Stochastic rounding: implementation, error analysis and applications
- Author
-
Matteo Croci, Massimiliano Fasi, Nicholas J. Higham, Theo Mary, Mantas Mikaitis, Mary, Theo, Plateforme d'analyse pour l'arithmétique flottante - - INTERFLOP2020 - ANR-20-CE46-0009 - AAPG2020 - VALID, University of Oxford [Oxford], Durham University, University of Manchester [Manchester], Performance et Qualité des Algorithmes Numériques (PEQUAN), LIP6, Sorbonne Université (SU)-Centre National de la Recherche Scientifique (CNRS)-Sorbonne Université (SU)-Centre National de la Recherche Scientifique (CNRS), and ANR-20-CE46-0009,INTERFLOP,Plateforme d'analyse pour l'arithmétique flottante(2020)
- Subjects
Multidisciplinary ,Computer Science Floating-point Arithmetic ,Partial Differential Equations ,[MATH] Mathematics [math] ,[INFO] Computer Science [cs] ,Rounding Error Analysis ,Ordinary Differential Equations ,IEEE 754 ,[INFO]Computer Science [cs] ,Computer Arithmetic ,Low Precision ,[MATH]Mathematics [math] ,binary16 ,bfloat16 ,Mathematics - Abstract
Stochastic rounding (SR) randomly maps a real number x to one of the two nearest values in a finite precision number system. The probability of choosing either of these two numbers is 1 minus their relative distance to x . This rounding mode was first proposed for use in computer arithmetic in the 1950s and it is currently experiencing a resurgence of interest. If used to compute the inner product of two vectors of length n in floating-point arithmetic, it yields an error bound with constant n u with high probability, where u is the unit round-off. This is not necessarily the case for round to nearest (RN), for which the worst-case error bound has constant nu . A particular attraction of SR is that, unlike RN, it is immune to the phenomenon of stagnation, whereby a sequence of tiny updates to a relatively large quantity is lost. We survey SR by discussing its mathematical properties and probabilistic error analysis, its implementation, and its use in applications, with a focus on machine learning and the numerical solution of differential equations.
- Published
- 2022