Juliacon 2024

Everybody wants fast code, but not everyone knows how to write it. Thankfully, Dagger.jl can help you write fast, parallel code with such ease as you've never known before. In this workshop, we will teach you everything you need to know to use Dagger effectively, and work through building example parallel programs with Dagger.

Large Language Models (LLMs) have become ubiquitous in several areas. But so far, the Julia ecosystem has lacked an efficient way to train these models, hindering the adoption of Julia by ML practitioners and users. In this talk we demonstrate parallel training of LLMs using Dagger.jl and Flux.jl. We discuss the various components required to write an efficient parallel training routine. Further, we present the scaling performance achievable with this method, and discuss the future developments.

Task parallelism is a simple but powerful method to make programs and libraries scalable, but which task parallel library you choose greatly affects how well and far your code can scale. In this talk, we will discuss the Dagger.jl package, which implements a variety of powerful abstractions on top of a high-performance, heterogeneous task parallel core.