2025-10-02 –, Robert Faure Amphitheater
Language: English
In Float64, 0.1 + 0.2 – 0.3 is not zero due to round-off errors. Try it. Stochastic rounding provides a trick: floating point rounding is done probabilistically: sometimes up, sometimes down. This allows for numerical simulations on hardware made for machine learning, where Float16 precision is common. We can perform meaningful simulations even with limited bits.
And yes, all of this can be done in Julia.
In standard floating point (FP) arithmetic, the simple expression 0.2 + 0.1 - 0.3 never evaluates to zero. Try it now in your REPL. This happens due to numerical round-off errors, as specified by IEEE 754. But what if the expression is zero on average? This is where stochastic rounding (SR) comes into play.
In a nutshell, SR decides after each FP operation whether to round up or down to fit the exact result into the FP type. The probability depends on the distance to the next available larger or smaller FP. The topic reappeared recently in research in the context of machine learning. New specialized hardware can perform huge parallel computations in low precision, down to Float16.
SR compensates for numerical errors in low precision, making it applicable to numerical simulations, as shown in [1,2]. StochasticRounding.jl [3] provides a Julia implementation of stochastic FP types, where the effects of stochastic rounding can be investigated. However, since this is done in software, it is slow. Nevertheless, a first processor unit that performs SR in silicon is available [4]. And yes, we can run Julia code on such machines [5]. There is industrial interest, and other vendors may follow to implement SR in their hardware.
This talk will provide an overview of the mathematical properties of SR, the implementation in Julia, as well as showcase the potential of running large-scale simulations at low precision.
[1] Klöwer, M. et al. Periodic orbits in chaotic systems simulated at low precision. Sci Rep 13, 11410 (2023). https://doi.org/10.1038/s41598-023-37004-4
[2] Paxton, E. et al. Climate Modeling in Low Precision: Effects of Both Deterministic and Stochastic Rounding. Journal of Climate, 2022, http://dx.doi.org/10.1175/JCLI-D-21-0343.1
[3] https://github.com/milankl/StochasticRounding.jl
[4] https://www.graphcore.ai/products/ipu
[5] https://giordano.github.io/talks/2024-06-20-julia-ipu-ornl
I'm a postdoc researcher at the Weierstrass Institute in Berlin in the scientific computing group.
My day-to-day work is all about running finite element simulations for solid mechanics and quantum devices.
I came into touch with Julia two years ago and I love it since.
At WIAS, we have our own Julia PDE solver ecosystem WIAS-PDELib, where I am a core maintainer.
I'm interested in low-level solver routines, software architecture, coding quality, and experiments with new data types and programming features.