Arkadii
Arkadii Bessonov is an LLM Engineer specializing in large-scale model training infrastructure. His work focuses on production-scale pretraining.
Session
LLM training demands ever more GPU-hours and memory, making any savings extremely valuable. As models grow, BF16 is increasingly expensive in both compute and memory. FP8 is rapidly becoming the standard for large-scale pretraining, yet it remains a mystery to most practitioners due to the numerical tricks it requires.
This talk demystifies FP8 training, drawing from hands-on experience implementing FP8 in large-scale production LLM pretraining: what happens under the hood of libraries like Transformer Engine and torchao, why a naive approach breaks training, and what practical approaches keep it stable — including scaling strategies, format choices, and selective higher-precision fallbacks. We also take a brief look at advanced techniques pushing FP8 beyond matrix multiplications.
You will leave with a clear understanding of how FP8 training works, what can go wrong, and how to use it in your pipeline.
Anyone interested in modern LLM pretraining is welcome. This talk will be especially relevant for LLM practitioners and performance engineers.